2024-05-08T15:35:35.714 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T15:35:35.714 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_lflores@teuthology --archive /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604 --verbose --description rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} --name lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi -- /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/orig.config.yaml 2024-05-08T15:35:35.755 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T15:35:35.847 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604 branch: wip-lflores-testing-2-2024-05-07-1606-squid description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} email: null first_in_suite: false job_id: '7697604' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lflores-testing-2-2024-05-07-1606-squid ceph: conf: client: debug monc: 20 debug ms: 1 global: mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr2: false ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 25 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub inject crc mismatch: 0.01 mon scrub inject missing keys: 0.05 mon scrub interval: 300 paxos min: 10 paxos service trim min: 5 paxos trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - ScrubResult - scrub mismatch - overall HEALTH_ - \(MGR_DOWN\) - \(PG_AVAILABILITY\) - \(SLOW_OPS\) - slow request - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(POOL_APP_NOT_ENABLED\) - slow request - overall HEALTH_ mon_bind_msgr2: false sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lflores-testing-2-2024-05-07-1606-squid sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 owner: scheduled_lflores@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mon.d - mon.e - osd.0 - osd.1 - osd.2 - - mon.f - mon.g - mon.h - mon.i - mgr.x - osd.3 - osd.4 - osd.5 - client.0 seed: 8082 sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-lflores-testing-2-2024-05-07-1606-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBczkMex5Mf99nGnIe00Fz2HEdqIoPkZbBaLijHg9c3JfJMQs1aH+scqmEQNI4/ZZWo9ylBnK+JzYTCwy8VbF5M= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFlyhAXJ2YIeYFuR/c5NEwAK94zifa9NMoYi8r2HVrS51RR0Z9byk2Kir2kWlPXPg8sj/61QiM+IolxhdGuhX8= tasks: - install: null - ceph: null - mon_thrash: revive_delay: 90 thrash_delay: 1 thrash_many: true - exec: client.0: - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel - ceph_test_rados_delete_pools_parallel teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_14:59:36 tube: smithi user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T15:35:35.847 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa; will attempt to use it 2024-05-08T15:35:35.848 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa/tasks 2024-05-08T15:35:35.848 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T15:35:35.850 INFO:teuthology.task.internal:Checking packages... 2024-05-08T15:35:35.874 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '95a4ad3c90f2f12f920e87725ad8d9002f948722' 2024-05-08T15:35:35.874 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T15:35:35.874 INFO:teuthology.packaging:ref: None 2024-05-08T15:35:35.874 INFO:teuthology.packaging:tag: None 2024-05-08T15:35:35.874 INFO:teuthology.packaging:branch: wip-lflores-testing-2-2024-05-07-1606-squid 2024-05-08T15:35:35.874 INFO:teuthology.packaging:sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T15:35:35.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lflores-testing-2-2024-05-07-1606-squid 2024-05-08T15:35:36.203 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2517.g95a4ad3c 2024-05-08T15:35:36.204 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T15:35:36.213 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T15:35:36.214 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T15:35:36.231 INFO:teuthology.task.internal:Saving configuration 2024-05-08T15:35:36.248 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T15:35:36.253 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T15:35:36.275 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 15:30:33.038072', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBczkMex5Mf99nGnIe00Fz2HEdqIoPkZbBaLijHg9c3JfJMQs1aH+scqmEQNI4/ZZWo9ylBnK+JzYTCwy8VbF5M='} 2024-05-08T15:35:36.299 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 15:30:33.039383', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFlyhAXJ2YIeYFuR/c5NEwAK94zifa9NMoYi8r2HVrS51RR0Z9byk2Kir2kWlPXPg8sj/61QiM+IolxhdGuhX8='} 2024-05-08T15:35:36.299 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T15:35:36.317 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'] 2024-05-08T15:35:36.317 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0'] 2024-05-08T15:35:36.317 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T15:35:36.388 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ca6182e50>, signals=[15]) 2024-05-08T15:35:36.389 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T15:35:36.395 INFO:teuthology.task.internal:Opening connections... 2024-05-08T15:35:36.395 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:35:36.397 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:35:36.479 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:35:36.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:35:36.553 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T15:35:36.559 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-05-08T15:35:36.578 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-05-08T15:35:36.578 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-05-08T15:35:36.638 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-05-08T15:35:36.638 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-05-08T15:35:36.638 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-05-08T15:35:36.638 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-05-08T15:35:36.638 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T15:35:36.639 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T15:35:36.640 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-05-08T15:35:36.671 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-05-08T15:35:36.688 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-05-08T15:35:36.689 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-05-08T15:35:36.745 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-05-08T15:35:36.745 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-05-08T15:35:36.745 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-05-08T15:35:36.745 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-05-08T15:35:36.745 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T15:35:36.746 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T15:35:36.747 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-05-08T15:35:36.770 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T15:35:36.780 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T15:35:36.785 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T15:35:36.785 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-05-08T15:35:36.788 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-05-08T15:35:36.806 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T15:35:36.812 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T15:35:36.813 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-05-08T15:35:36.848 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-05-08T15:35:36.921 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T15:35:36.978 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T15:35:36.993 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T15:35:36.993 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.f': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T15:35:36.993 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T15:35:36.994 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T15:35:37.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:35:37.011 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-05-08T15:35:37.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:35:37.012 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-05-08T15:35:37.069 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-437.el9.x86_64 2024-05-08T15:35:37.069 INFO:teuthology.task.kernel:Running kernel on smithi081: 5.14.0-437.el9.x86_64 2024-05-08T15:35:37.069 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum install -y kernel 2024-05-08T15:35:37.070 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-437.el9.x86_64 2024-05-08T15:35:37.070 INFO:teuthology.task.kernel:Running kernel on smithi007: 5.14.0-437.el9.x86_64 2024-05-08T15:35:37.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum install -y kernel 2024-05-08T15:35:40.667 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.1 MB 00:01 2024-05-08T15:35:42.468 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-08T15:35:43.349 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.1 MB 00:04 2024-05-08T15:35:46.843 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-08T15:35:48.002 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-08T15:35:49.502 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-08T15:35:51.579 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-05-08T15:35:52.393 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:02 2024-05-08T15:35:53.819 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-05-08T15:35:56.912 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-08T15:35:59.060 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T15:35:59.060 INFO:teuthology.orchestra.run.smithi007.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T15:35:59.160 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:35:59.172 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:35:59.173 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-05-08T15:35:59.174 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:35:59.174 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 78 M 2024-05-08T15:35:59.174 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 123 M 2024-05-08T15:35:59.174 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:36:00.288 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 485 kB/s | 484 kB 00:00 2024-05-08T15:36:00.980 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 9.1 MB/s | 15 MB 00:01 2024-05-08T15:36:01.672 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-05-08T15:36:02.622 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 12 MB/s | 28 MB 00:02 2024-05-08T15:36:02.622 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:36:02.623 INFO:teuthology.orchestra.run.smithi007.stdout:Total 23 MB/s | 78 MB 00:03 2024-05-08T15:36:03.333 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:36:03.769 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:36:03.769 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:36:04.715 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:36:04.715 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:36:04.877 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 21 MB 00:10 2024-05-08T15:36:07.266 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:36:07.922 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T15:36:08.006 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T15:36:10.135 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-08T15:36:10.829 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T15:36:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T15:36:12.150 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T15:36:12.150 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T15:36:12.244 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:36:12.254 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:36:12.254 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-05-08T15:36:12.254 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout:Install 4 Packages 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 78 M 2024-05-08T15:36:12.255 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 123 M 2024-05-08T15:36:12.256 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:36:12.845 INFO:teuthology.orchestra.run.smithi081.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.5 MB/s | 484 kB 00:00 2024-05-08T15:36:14.287 INFO:teuthology.orchestra.run.smithi081.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 8.8 MB/s | 15 MB 00:01 2024-05-08T15:36:14.912 INFO:teuthology.orchestra.run.smithi081.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 14 MB/s | 28 MB 00:02 2024-05-08T15:36:15.288 INFO:teuthology.orchestra.run.smithi081.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 12 MB/s | 34 MB 00:02 2024-05-08T15:36:15.288 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:36:15.288 INFO:teuthology.orchestra.run.smithi081.stdout:Total 26 MB/s | 78 MB 00:03 2024-05-08T15:36:15.975 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:36:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T15:36:16.109 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:16.311 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:36:16.311 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:36:17.259 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:36:17.260 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:36:19.673 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:36:20.358 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T15:36:20.439 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T15:36:21.194 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:23.147 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T15:36:23.278 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T15:36:28.014 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T15:36:28.189 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:32.921 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:52.239 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:52.279 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T15:36:52.574 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T15:36:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T15:36:53.347 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:36:53.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:36:53.347 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:36:53.347 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T15:36:53.347 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T15:36:53.348 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T15:36:53.348 INFO:teuthology.orchestra.run.smithi007.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T15:36:53.348 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:36:53.348 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:36:53.712 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T15:36:53.791 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T15:36:53.794 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T15:36:53.794 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T15:36:53.794 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T15:36:53.794 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T15:36:53.794 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T15:36:53.794 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T15:36:53.794 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T15:36:53.845 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T15:36:53.845 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T15:36:53.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 68.el9 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8264254 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T15:36:53.877 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:36:53.878 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T15:36:53.879 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T15:36:53.879 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 5.14.0-444.el9.x86_64 2024-05-08T15:36:53.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T15:36:54.544 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-05-08T15:36:55.971 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T15:36:56.022 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-05-08T15:36:56.025 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-08T15:36:56.046 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.JGLmxARN2g 2024-05-08T15:36:56.047 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JGLmxARN2g 2024-05-08T15:36:56.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.JGLmxARN2g 2024-05-08T15:36:56.285 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.JGLmxARN2g is 6KB 2024-05-08T15:36:56.338 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.JGLmxARN2g 2024-05-08T15:36:56.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T15:36:56.422 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T15:36:56.423 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T15:36:56.423 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T15:36:56.423 INFO:teuthology.orchestra.run.smithi007.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T15:36:56.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T15:36:56.572 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-05-08T15:37:02.514 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:37:02.552 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:37:02.827 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:37:02.827 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T15:37:02.827 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T15:37:02.827 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:37:03.641 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:37:04.010 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T15:37:04.089 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T15:37:04.089 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T15:37:04.090 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T15:37:04.090 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T15:37:04.090 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T15:37:04.091 INFO:teuthology.task.kernel:Installing distro kernel on mon.f... 2024-05-08T15:37:04.091 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T15:37:04.091 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T15:37:04.141 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T15:37:04.141 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T15:37:04.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-05-08T15:37:04.178 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 68.el9 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8264254 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T15:37:04.179 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:37:04.180 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T15:37:04.181 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T15:37:04.181 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 5.14.0-444.el9.x86_64 2024-05-08T15:37:04.182 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T15:37:04.769 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-05-08T15:37:06.209 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T15:37:06.239 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-05-08T15:37:06.241 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-05-08T15:37:06.263 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.XhP0nDP5YC 2024-05-08T15:37:06.263 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XhP0nDP5YC 2024-05-08T15:37:06.329 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.XhP0nDP5YC 2024-05-08T15:37:06.465 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.XhP0nDP5YC is 6KB 2024-05-08T15:37:06.514 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.XhP0nDP5YC 2024-05-08T15:37:06.529 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T15:37:06.589 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T15:37:06.590 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T15:37:06.590 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T15:37:06.590 INFO:teuthology.orchestra.run.smithi081.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T15:37:06.591 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T15:37:06.744 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-05-08T15:37:26.599 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T15:37:26.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-08T15:37:26.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:37:36.749 INFO:teuthology.task.kernel:Checking client mon.f for new kernel version... 2024-05-08T15:37:36.749 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T15:37:36.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:37:44.990 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-05-08T15:37:53.998 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-08T15:37:53.999 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:37:55.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-05-08T15:38:04.238 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T15:38:04.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:38:07.298 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-05-08T15:38:19.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T15:38:19.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:38:37.730 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-05-08T15:38:52.742 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T15:38:52.743 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:38:53.058 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-05-08T15:38:53.463 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T15:38:53.463 INFO:teuthology.task.kernel:Checking kernel version of mon.f, want "5.14.0-444.el9.x86_64"... 2024-05-08T15:38:53.463 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-05-08T15:38:53.477 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-444.el9.x86_64 2024-05-08T15:38:53.478 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T15:38:53.478 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T15:38:53.478 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T15:38:54.001 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T15:38:54.479 INFO:teuthology.task.kernel:Enabling kdb on mon.f... 2024-05-08T15:38:54.479 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T15:38:54.565 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-05-08T15:38:54.599 DEBUG:teuthology.parallel:result is None 2024-05-08T15:39:06.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-08T15:39:06.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:39:06.304 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-05-08T15:39:06.696 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-08T15:39:06.697 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-08T15:39:06.697 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-05-08T15:39:06.714 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-444.el9.x86_64 2024-05-08T15:39:06.715 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T15:39:06.715 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T15:39:06.715 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T15:39:07.716 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T15:39:07.716 DEBUG:teuthology.orchestra.run.smithi007:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T15:39:07.792 INFO:teuthology.orchestra.run.smithi007.stdout:ttyS1 2024-05-08T15:39:07.831 DEBUG:teuthology.parallel:result is None 2024-05-08T15:39:07.831 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T15:39:07.839 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T15:39:07.839 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T15:39:07.842 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T15:39:07.862 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T15:39:07.870 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T15:39:07.877 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T15:39:07.878 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T15:39:07.902 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T15:39:08.002 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T15:39:08.009 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T15:39:08.009 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T15:39:08.011 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T15:39:08.042 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T15:39:08.044 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T15:39:08.055 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T15:39:08.056 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T15:39:08.058 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T15:39:08.066 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T15:39:08.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T15:39:08.102 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T15:39:08.131 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T15:39:08.140 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T15:39:08.141 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T15:39:08.173 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T15:39:08.190 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T15:39:08.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T15:39:08.334 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T15:39:08.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T15:39:08.462 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:39:08.462 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T15:39:08.529 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T15:39:08.574 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T15:39:08.629 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T15:39:08.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T15:39:08.757 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:39:08.757 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T15:39:08.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-05-08T15:39:08.827 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-05-08T15:39:08.870 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T15:39:08.910 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T15:39:09.373 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T15:39:09.431 INFO:teuthology.task.internal:Starting timer... 2024-05-08T15:39:09.431 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T15:39:09.463 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T15:39:09.475 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T15:39:09.475 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-05-08T15:39:09.549 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-05-08T15:39:09.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-05-08T15:39:09.648 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-05-08T15:39:09.651 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T15:39:09.651 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-05-08T15:39:09.689 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-05-08T15:39:09.689 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-05-08T15:39:09.714 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-05-08T15:39:09.714 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive'} 2024-05-08T15:39:09.715 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T15:39:09.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:39:09.759 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T15:39:09.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:39:09.787 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T15:39:09.788 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-05-08T15:39:09.828 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-05-08T15:39:09.858 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T15:39:09.867 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T15:39:09.885 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T15:39:09.886 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi081.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T15:43:18.802 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com')] 2024-05-08T15:43:18.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-08T15:43:18.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:43:18.888 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-05-08T15:43:18.966 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-05-08T15:43:18.967 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T15:43:18.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T15:43:19.042 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-05-08T15:43:19.117 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-05-08T15:43:19.117 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T15:43:19.128 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T15:43:19.128 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T15:43:19.129 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T15:43:19.131 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T15:43:19.132 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T15:43:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T15:43:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T15:43:19.186 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T15:43:19.200 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T15:43:19.201 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-05-08T15:43:19.213 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-05-08T15:43:19.224 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-05-08T15:43:19.228 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T15:43:19.235 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-05-08T15:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T15:43:19.248 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T15:43:19.262 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T15:43:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-05-08T15:43:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T15:43:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-05-08T15:43:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.304 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.314 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-05-08T15:43:19.318 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T15:43:19.318 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-05-08T15:43:19.318 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.318 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.319 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.319 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:43:19.319 INFO:teuthology.run_tasks:Running task install... 2024-05-08T15:43:19.334 DEBUG:teuthology.task.install:project ceph 2024-05-08T15:43:19.334 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}} 2024-05-08T15:43:19.334 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'} 2024-05-08T15:43:19.335 INFO:teuthology.task.install:Using flavor: default 2024-05-08T15:43:19.341 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T15:43:19.341 INFO:teuthology.task.install:extra packages: [] 2024-05-08T15:43:19.341 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'tag': None, 'wait_for_package': False} 2024-05-08T15:43:19.342 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T15:43:19.343 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'tag': None, 'wait_for_package': False} 2024-05-08T15:43:19.344 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T15:43:19.501 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/ 2024-05-08T15:43:19.501 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2517.g95a4ad3c 2024-05-08T15:43:19.642 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/ 2024-05-08T15:43:19.642 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2517.g95a4ad3c 2024-05-08T15:43:19.723 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T15:43:19.723 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:43:19.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T15:43:19.752 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T15:43:19.753 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/95a4ad3c90f2f12f920e87725ad8d9002f948722/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T15:43:19.821 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T15:43:19.838 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T15:43:19.839 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:43:19.839 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T15:43:19.865 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T15:43:19.865 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/95a4ad3c90f2f12f920e87725ad8d9002f948722/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T15:43:19.898 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T15:43:19.931 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T15:43:19.963 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-05-08T15:43:19.965 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-05-08T15:43:20.006 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T15:43:20.070 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2024-05-08T15:43:20.072 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2024-05-08T15:43:20.204 INFO:teuthology.orchestra.run.smithi007.stdout:54 files removed 2024-05-08T15:43:20.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw 2024-05-08T15:43:20.313 INFO:teuthology.orchestra.run.smithi081.stdout:54 files removed 2024-05-08T15:43:20.342 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw 2024-05-08T15:43:20.892 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 223 kB/s | 83 kB 00:00 2024-05-08T15:43:21.029 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 200 kB/s | 83 kB 00:00 2024-05-08T15:43:21.434 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-05-08T15:43:21.563 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-05-08T15:43:21.918 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-05-08T15:43:21.938 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-05-08T15:43:23.451 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.1 MB 00:01 2024-05-08T15:43:24.588 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.1 MB 00:02 2024-05-08T15:43:26.727 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 19 MB 00:02 2024-05-08T15:43:27.999 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-05-08T15:43:30.912 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-08T15:43:32.630 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-05-08T15:43:33.408 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-05-08T15:43:34.834 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-08T15:43:35.117 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-08T15:43:36.977 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-08T15:43:40.139 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-08T15:43:42.094 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-08T15:43:42.634 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 10 M 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.4 M 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.4 M 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 21 M 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 25 k 2024-05-08T15:43:42.639 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 745 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 501 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.3 M 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 45 k 2024-05-08T15:43:42.640 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 135 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 162 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 320 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 299 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 99 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:43:42.641 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:43:42.642 INFO:teuthology.orchestra.run.smithi007.stdout:Install 30 Packages 2024-05-08T15:43:42.642 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 2 Packages 2024-05-08T15:43:42.642 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:43:42.643 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 60 M 2024-05-08T15:43:42.643 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:43:43.866 INFO:teuthology.orchestra.run.smithi007.stdout:(1/32): ceph-base-19.0.0-2517.g95a4ad3c.el9.x86 5.7 MB/s | 5.4 MB 00:00 2024-05-08T15:43:43.908 INFO:teuthology.orchestra.run.smithi007.stdout:(2/32): ceph-selinux-19.0.0-2517.g95a4ad3c.el9. 609 kB/s | 25 kB 00:00 2024-05-08T15:43:44.025 INFO:teuthology.orchestra.run.smithi007.stdout:(3/32): libcephfs2-19.0.0-2517.g95a4ad3c.el9.x8 6.2 MB/s | 745 kB 00:00 2024-05-08T15:43:44.125 INFO:teuthology.orchestra.run.smithi007.stdout:(4/32): libradosstriper1-19.0.0-2517.g95a4ad3c. 4.9 MB/s | 501 kB 00:00 2024-05-08T15:43:44.451 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 10 M 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.4 M 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T15:43:44.469 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.4 M 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 21 M 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 25 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 745 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 501 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.3 M 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T15:43:44.470 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 45 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 135 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 162 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 320 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 299 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 99 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T15:43:44.471 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:43:44.472 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:43:44.472 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:43:44.472 INFO:teuthology.orchestra.run.smithi081.stdout:Install 30 Packages 2024-05-08T15:43:44.472 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 2 Packages 2024-05-08T15:43:44.472 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:43:44.472 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 60 M 2024-05-08T15:43:44.472 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:43:44.892 INFO:teuthology.orchestra.run.smithi007.stdout:(5/32): librgw2-19.0.0-2517.g95a4ad3c.el9.x86_6 6.9 MB/s | 5.3 MB 00:00 2024-05-08T15:43:44.959 INFO:teuthology.orchestra.run.smithi007.stdout:(6/32): python3-ceph-argparse-19.0.0-2517.g95a4 679 kB/s | 45 kB 00:00 2024-05-08T15:43:45.209 INFO:teuthology.orchestra.run.smithi007.stdout:(7/32): ceph-common-19.0.0-2517.g95a4ad3c.el9.x 9.2 MB/s | 21 MB 00:02 2024-05-08T15:43:45.226 INFO:teuthology.orchestra.run.smithi007.stdout:(8/32): python3-ceph-common-19.0.0-2517.g95a4ad 506 kB/s | 135 kB 00:00 2024-05-08T15:43:45.260 INFO:teuthology.orchestra.run.smithi007.stdout:(9/32): python3-cephfs-19.0.0-2517.g95a4ad3c.el 3.2 MB/s | 162 kB 00:00 2024-05-08T15:43:45.310 INFO:teuthology.orchestra.run.smithi007.stdout:(10/32): python3-rados-19.0.0-2517.g95a4ad3c.el 3.7 MB/s | 320 kB 00:00 2024-05-08T15:43:45.344 INFO:teuthology.orchestra.run.smithi007.stdout:(11/32): python3-rbd-19.0.0-2517.g95a4ad3c.el9. 3.5 MB/s | 299 kB 00:00 2024-05-08T15:43:45.369 INFO:teuthology.orchestra.run.smithi007.stdout:(12/32): python3-rgw-19.0.0-2517.g95a4ad3c.el9. 1.7 MB/s | 99 kB 00:00 2024-05-08T15:43:45.477 INFO:teuthology.orchestra.run.smithi007.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 801 kB/s | 107 kB 00:00 2024-05-08T15:43:45.503 INFO:teuthology.orchestra.run.smithi007.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-08T15:43:45.528 INFO:teuthology.orchestra.run.smithi007.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-05-08T15:43:45.662 INFO:teuthology.orchestra.run.smithi007.stdout:(16/32): ceph-radosgw-19.0.0-2517.g95a4ad3c.el9 3.7 MB/s | 10 MB 00:02 2024-05-08T15:43:45.688 INFO:teuthology.orchestra.run.smithi007.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-05-08T15:43:45.712 INFO:teuthology.orchestra.run.smithi007.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-05-08T15:43:45.738 INFO:teuthology.orchestra.run.smithi007.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-08T15:43:45.755 INFO:teuthology.orchestra.run.smithi007.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 857 kB/s | 36 kB 00:00 2024-05-08T15:43:45.772 INFO:teuthology.orchestra.run.smithi007.stdout:(21/32): python3-prettytable-0.7.2-27.el9.noarc 387 kB/s | 42 kB 00:00 2024-05-08T15:43:45.789 INFO:teuthology.orchestra.run.smithi007.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 937 kB/s | 47 kB 00:00 2024-05-08T15:43:45.887 INFO:teuthology.orchestra.run.smithi081.stdout:(1/32): ceph-base-19.0.0-2517.g95a4ad3c.el9.x86 5.4 MB/s | 5.4 MB 00:00 2024-05-08T15:43:45.937 INFO:teuthology.orchestra.run.smithi081.stdout:(2/32): ceph-selinux-19.0.0-2517.g95a4ad3c.el9. 505 kB/s | 25 kB 00:00 2024-05-08T15:43:45.956 INFO:teuthology.orchestra.run.smithi007.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-05-08T15:43:46.014 INFO:teuthology.orchestra.run.smithi007.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-05-08T15:43:46.031 INFO:teuthology.orchestra.run.smithi007.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-05-08T15:43:46.054 INFO:teuthology.orchestra.run.smithi081.stdout:(3/32): libcephfs2-19.0.0-2517.g95a4ad3c.el9.x8 6.3 MB/s | 745 kB 00:00 2024-05-08T15:43:46.073 INFO:teuthology.orchestra.run.smithi007.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-08T15:43:46.124 INFO:teuthology.orchestra.run.smithi007.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 3.8 MB/s | 192 kB 00:00 2024-05-08T15:43:46.146 INFO:teuthology.orchestra.run.smithi081.stdout:(4/32): libradosstriper1-19.0.0-2517.g95a4ad3c. 5.3 MB/s | 501 kB 00:00 2024-05-08T15:43:46.182 INFO:teuthology.orchestra.run.smithi007.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 839 kB 00:00 2024-05-08T15:43:46.266 INFO:teuthology.orchestra.run.smithi007.stdout:(29/32): libarrow-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 4.4 MB 00:00 2024-05-08T15:43:46.341 INFO:teuthology.orchestra.run.smithi007.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 7.3 MB/s | 1.6 MB 00:00 2024-05-08T15:43:46.947 INFO:teuthology.orchestra.run.smithi081.stdout:(5/32): librgw2-19.0.0-2517.g95a4ad3c.el9.x86_6 6.6 MB/s | 5.3 MB 00:00 2024-05-08T15:43:46.992 INFO:teuthology.orchestra.run.smithi007.stdout:(31/32): librbd1-19.0.0-2517.g95a4ad3c.el9.x86_ 4.3 MB/s | 3.1 MB 00:00 2024-05-08T15:43:47.013 INFO:teuthology.orchestra.run.smithi081.stdout:(6/32): python3-ceph-argparse-19.0.0-2517.g95a4 677 kB/s | 45 kB 00:00 2024-05-08T15:43:47.080 INFO:teuthology.orchestra.run.smithi081.stdout:(7/32): python3-ceph-common-19.0.0-2517.g95a4ad 2.0 MB/s | 135 kB 00:00 2024-05-08T15:43:47.225 INFO:teuthology.orchestra.run.smithi007.stdout:(32/32): librados2-19.0.0-2517.g95a4ad3c.el9.x8 3.2 MB/s | 3.4 MB 00:01 2024-05-08T15:43:47.230 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:43:47.231 INFO:teuthology.orchestra.run.smithi007.stdout:Total 13 MB/s | 60 MB 00:04 2024-05-08T15:43:47.339 INFO:teuthology.orchestra.run.smithi081.stdout:(8/32): ceph-common-19.0.0-2517.g95a4ad3c.el9.x 8.6 MB/s | 21 MB 00:02 2024-05-08T15:43:47.341 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:43:47.364 INFO:teuthology.orchestra.run.smithi081.stdout:(9/32): python3-cephfs-19.0.0-2517.g95a4ad3c.el 571 kB/s | 162 kB 00:00 2024-05-08T15:43:47.383 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:43:47.383 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:43:47.406 INFO:teuthology.orchestra.run.smithi081.stdout:(10/32): python3-rados-19.0.0-2517.g95a4ad3c.el 4.7 MB/s | 320 kB 00:00 2024-05-08T15:43:47.432 INFO:teuthology.orchestra.run.smithi081.stdout:(11/32): python3-rbd-19.0.0-2517.g95a4ad3c.el9. 4.4 MB/s | 299 kB 00:00 2024-05-08T15:43:47.457 INFO:teuthology.orchestra.run.smithi081.stdout:(12/32): python3-rgw-19.0.0-2517.g95a4ad3c.el9. 1.9 MB/s | 99 kB 00:00 2024-05-08T15:43:47.566 INFO:teuthology.orchestra.run.smithi081.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 801 kB/s | 107 kB 00:00 2024-05-08T15:43:47.608 INFO:teuthology.orchestra.run.smithi081.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-05-08T15:43:47.633 INFO:teuthology.orchestra.run.smithi081.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 676 kB/s | 45 kB 00:00 2024-05-08T15:43:47.691 INFO:teuthology.orchestra.run.smithi081.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-05-08T15:43:47.717 INFO:teuthology.orchestra.run.smithi081.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-08T15:43:47.746 INFO:teuthology.orchestra.run.smithi081.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 841 kB/s | 42 kB 00:00 2024-05-08T15:43:47.767 INFO:teuthology.orchestra.run.smithi081.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-08T15:43:47.792 INFO:teuthology.orchestra.run.smithi081.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 714 kB/s | 36 kB 00:00 2024-05-08T15:43:47.818 INFO:teuthology.orchestra.run.smithi081.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-05-08T15:43:48.010 INFO:teuthology.orchestra.run.smithi081.stdout:(22/32): ceph-radosgw-19.0.0-2517.g95a4ad3c.el9 3.3 MB/s | 10 MB 00:03 2024-05-08T15:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:43:48.047 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:43:48.202 INFO:teuthology.orchestra.run.smithi081.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 755 kB/s | 309 kB 00:00 2024-05-08T15:43:48.227 INFO:teuthology.orchestra.run.smithi081.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 121 kB/s | 26 kB 00:00 2024-05-08T15:43:48.269 INFO:teuthology.orchestra.run.smithi081.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 727 kB/s | 49 kB 00:00 2024-05-08T15:43:48.311 INFO:teuthology.orchestra.run.smithi081.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-05-08T15:43:48.453 INFO:teuthology.orchestra.run.smithi081.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-05-08T15:43:48.520 INFO:teuthology.orchestra.run.smithi081.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 839 kB 00:00 2024-05-08T15:43:48.912 INFO:teuthology.orchestra.run.smithi081.stdout:(29/32): libarrow-9.0.0-11.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-05-08T15:43:48.954 INFO:teuthology.orchestra.run.smithi081.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-05-08T15:43:49.046 INFO:teuthology.orchestra.run.smithi081.stdout:(31/32): librados2-19.0.0-2517.g95a4ad3c.el9.x8 6.4 MB/s | 3.4 MB 00:00 2024-05-08T15:43:49.288 INFO:teuthology.orchestra.run.smithi081.stdout:(32/32): librbd1-19.0.0-2517.g95a4ad3c.el9.x86_ 8.4 MB/s | 3.1 MB 00:00 2024-05-08T15:43:49.294 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:43:49.294 INFO:teuthology.orchestra.run.smithi081.stdout:Total 12 MB/s | 60 MB 00:04 2024-05-08T15:43:49.399 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:43:49.435 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:43:49.436 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:43:49.475 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:43:49.550 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T15:43:49.581 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T15:43:49.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T15:43:49.636 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T15:43:49.662 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T15:43:49.830 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T15:43:49.886 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T15:43:50.094 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:43:50.094 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:43:50.305 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T15:43:50.361 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T15:43:50.451 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T15:43:50.541 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 8/34 2024-05-08T15:43:50.567 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T15:43:50.631 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T15:43:50.668 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T15:43:50.713 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 11/34 2024-05-08T15:43:50.782 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T15:43:50.803 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T15:43:50.868 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T15:43:50.902 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T15:43:51.014 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T15:43:51.045 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T15:43:51.107 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T15:43:51.135 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T15:43:51.164 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T15:43:51.197 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T15:43:51.509 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:43:51.527 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T15:43:51.560 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T15:43:51.598 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T15:43:51.660 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T15:43:51.661 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T15:43:51.677 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T15:43:51.682 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T15:43:51.706 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 23/34 2024-05-08T15:43:51.707 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T15:43:51.743 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T15:43:51.851 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T15:43:51.905 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T15:43:52.240 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T15:43:52.271 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T15:43:52.292 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T15:43:52.335 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T15:43:52.353 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T15:43:52.415 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 27/34 2024-05-08T15:43:52.429 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T15:43:52.449 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 28/34 2024-05-08T15:43:52.519 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 8/34 2024-05-08T15:43:52.547 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T15:43:52.610 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T15:43:52.647 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T15:43:52.691 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 11/34 2024-05-08T15:43:52.761 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T15:43:52.783 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T15:43:52.853 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T15:43:52.882 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T15:43:52.992 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T15:43:53.024 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T15:43:53.066 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T15:43:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T15:43:53.149 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T15:43:53.184 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T15:43:53.503 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T15:43:53.547 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T15:43:53.638 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T15:43:53.669 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 23/34 2024-05-08T15:43:53.705 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T15:43:54.176 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T15:43:54.200 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T15:43:54.272 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T15:43:54.349 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 27/34 2024-05-08T15:43:54.378 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 28/34 2024-05-08T15:43:54.874 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T15:43:54.920 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T15:43:55.508 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T15:43:55.550 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T15:43:55.649 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T15:43:55.650 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T15:43:55.650 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T15:43:55.650 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:43:55.693 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T15:43:56.647 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T15:43:56.700 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T15:43:57.257 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T15:43:57.295 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T15:43:57.376 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T15:43:57.376 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T15:43:57.376 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T15:43:57.376 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:43:57.405 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T15:44:05.583 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T15:44:05.583 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-05-08T15:44:05.583 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-05-08T15:44:05.583 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-05-08T15:44:05.584 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-05-08T15:44:05.584 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-05-08T15:44:05.584 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-05-08T15:44:05.584 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-05-08T15:44:05.584 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:06.565 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T15:44:06.591 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T15:44:06.591 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:06.591 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T15:44:06.591 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T15:44:06.591 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T15:44:06.591 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:06.592 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T15:44:06.683 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T15:44:06.683 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T15:44:06.820 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2024-05-08T15:44:06.821 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:07.809 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T15:44:07.835 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T15:44:07.835 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:07.835 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T15:44:07.835 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T15:44:07.835 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T15:44:07.835 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:07.837 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T15:44:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T15:44:07.856 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 6/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 8/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 9/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 10/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 11/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 13/34 2024-05-08T15:44:07.857 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T15:44:07.858 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T15:44:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T15:44:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T15:44:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T15:44:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T15:44:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T15:44:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T15:44:07.859 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/34 2024-05-08T15:44:07.920 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T15:44:07.920 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T15:44:08.658 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T15:44:08.659 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:08.661 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:09.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-test 2024-05-08T15:44:09.074 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T15:44:09.074 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/34 2024-05-08T15:44:09.074 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/34 2024-05-08T15:44:09.074 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/34 2024-05-08T15:44:09.074 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/34 2024-05-08T15:44:09.074 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 6/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 8/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 9/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 10/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 11/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 13/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T15:44:09.075 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T15:44:09.076 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T15:44:09.077 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T15:44:09.077 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T15:44:09.077 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T15:44:09.077 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T15:44:09.077 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/34 2024-05-08T15:44:09.604 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:29 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:44:09.729 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 48 M 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 163 k 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:09.730 INFO:teuthology.orchestra.run.smithi007.stdout:Install 4 Packages 2024-05-08T15:44:09.731 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 49 M 2024-05-08T15:44:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 207 M 2024-05-08T15:44:09.731 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.994 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.995 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:09.996 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:44:10.123 INFO:teuthology.orchestra.run.smithi007.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-05-08T15:44:10.165 INFO:teuthology.orchestra.run.smithi007.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-08T15:44:10.380 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-test 2024-05-08T15:44:10.848 INFO:teuthology.orchestra.run.smithi007.stdout:(3/4): libcephsqlite-19.0.0-2517.g95a4ad3c.el9. 178 kB/s | 163 kB 00:00 2024-05-08T15:44:10.951 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:28 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:44:11.071 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 48 M 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 163 k 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout:Install 4 Packages 2024-05-08T15:44:11.072 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:11.073 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 49 M 2024-05-08T15:44:11.073 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 207 M 2024-05-08T15:44:11.073 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:44:11.276 INFO:teuthology.orchestra.run.smithi081.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 305 kB 00:00 2024-05-08T15:44:11.318 INFO:teuthology.orchestra.run.smithi081.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-05-08T15:44:11.951 INFO:teuthology.orchestra.run.smithi081.stdout:(3/4): libcephsqlite-19.0.0-2517.g95a4ad3c.el9. 194 kB/s | 163 kB 00:00 2024-05-08T15:44:19.406 INFO:teuthology.orchestra.run.smithi007.stdout:(4/4): ceph-test-19.0.0-2517.g95a4ad3c.el9.x86_ 5.1 MB/s | 48 MB 00:09 2024-05-08T15:44:19.408 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:19.408 INFO:teuthology.orchestra.run.smithi007.stdout:Total 5.0 MB/s | 49 MB 00:09 2024-05-08T15:44:19.436 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:44:19.458 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:44:19.458 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:44:19.844 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:44:19.844 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:44:20.395 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:44:20.469 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T15:44:20.518 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T15:44:20.549 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T15:44:25.403 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T15:44:25.475 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T15:44:27.850 INFO:teuthology.orchestra.run.smithi081.stdout:(4/4): ceph-test-19.0.0-2517.g95a4ad3c.el9.x86_ 2.9 MB/s | 48 MB 00:16 2024-05-08T15:44:27.852 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:27.853 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.9 MB/s | 49 MB 00:16 2024-05-08T15:44:27.880 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T15:44:27.880 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/4 2024-05-08T15:44:27.881 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/4 2024-05-08T15:44:27.881 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T15:44:27.881 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:44:27.901 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:44:27.901 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:28.187 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:28.268 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:44:28.268 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:44:28.458 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph 2024-05-08T15:44:28.820 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:44:28.895 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T15:44:28.941 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T15:44:28.969 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T15:44:29.044 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:00:49 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:44:29.174 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:44:29.176 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:29.176 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 6.6 k 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 2.4 M 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 1.5 M 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 249 k 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 4.7 M 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 17 M 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T15:44:29.177 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T15:44:29.178 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:44:29.179 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:29.180 INFO:teuthology.orchestra.run.smithi007.stdout:Install 36 Packages 2024-05-08T15:44:29.180 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:29.180 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 30 M 2024-05-08T15:44:29.180 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 107 M 2024-05-08T15:44:29.180 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:44:30.135 INFO:teuthology.orchestra.run.smithi007.stdout:(1/36): ceph-19.0.0-2517.g95a4ad3c.el9.x86_64.r 22 kB/s | 6.6 kB 00:00 2024-05-08T15:44:31.601 INFO:teuthology.orchestra.run.smithi007.stdout:(2/36): ceph-mgr-19.0.0-2517.g95a4ad3c.el9.x86_ 866 kB/s | 1.5 MB 00:01 2024-05-08T15:44:31.952 INFO:teuthology.orchestra.run.smithi007.stdout:(3/36): ceph-mds-19.0.0-2517.g95a4ad3c.el9.x86_ 1.1 MB/s | 2.4 MB 00:02 2024-05-08T15:44:32.218 INFO:teuthology.orchestra.run.smithi007.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2517.g95a4 933 kB/s | 249 kB 00:00 2024-05-08T15:44:32.444 INFO:teuthology.orchestra.run.smithi007.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-08T15:44:32.578 INFO:teuthology.orchestra.run.smithi007.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 9.2 MB/s | 1.2 MB 00:00 2024-05-08T15:44:32.653 INFO:teuthology.orchestra.run.smithi007.stdout:(7/36): ceph-mon-19.0.0-2517.g95a4ad3c.el9.x86_ 1.9 MB/s | 4.7 MB 00:02 2024-05-08T15:44:32.678 INFO:teuthology.orchestra.run.smithi007.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-08T15:44:32.737 INFO:teuthology.orchestra.run.smithi007.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-05-08T15:44:32.795 INFO:teuthology.orchestra.run.smithi007.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-05-08T15:44:32.829 INFO:teuthology.orchestra.run.smithi007.stdout:(11/36): python3-pycparser-2.20-6.el9.noarch.rp 770 kB/s | 135 kB 00:00 2024-05-08T15:44:33.503 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T15:44:33.571 INFO:teuthology.orchestra.run.smithi007.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 243 kB/s | 188 kB 00:00 2024-05-08T15:44:33.592 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T15:44:33.621 INFO:teuthology.orchestra.run.smithi007.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 217 kB/s | 172 kB 00:00 2024-05-08T15:44:34.180 INFO:teuthology.orchestra.run.smithi007.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 57 kB/s | 35 kB 00:00 2024-05-08T15:44:34.255 INFO:teuthology.orchestra.run.smithi007.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 80 kB/s | 51 kB 00:00 2024-05-08T15:44:34.822 INFO:teuthology.orchestra.run.smithi007.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 65 kB/s | 42 kB 00:00 2024-05-08T15:44:34.858 INFO:teuthology.orchestra.run.smithi007.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 37 kB/s | 22 kB 00:00 2024-05-08T15:44:35.156 INFO:teuthology.orchestra.run.smithi007.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 145 kB/s | 43 kB 00:00 2024-05-08T15:44:35.198 INFO:teuthology.orchestra.run.smithi007.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 403 kB/s | 151 kB 00:00 2024-05-08T15:44:35.356 INFO:teuthology.orchestra.run.smithi007.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 2.2 MB/s | 358 kB 00:00 2024-05-08T15:44:35.382 INFO:teuthology.orchestra.run.smithi007.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 762 kB/s | 172 kB 00:00 2024-05-08T15:44:35.482 INFO:teuthology.orchestra.run.smithi007.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 85 kB/s | 11 kB 00:00 2024-05-08T15:44:35.507 INFO:teuthology.orchestra.run.smithi007.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2024-05-08T15:44:35.549 INFO:teuthology.orchestra.run.smithi007.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 346 kB/s | 23 kB 00:00 2024-05-08T15:44:35.583 INFO:teuthology.orchestra.run.smithi007.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 259 kB/s | 19 kB 00:00 2024-05-08T15:44:35.650 INFO:teuthology.orchestra.run.smithi007.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-05-08T15:44:35.683 INFO:teuthology.orchestra.run.smithi007.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 461 kB/s | 46 kB 00:00 2024-05-08T15:44:35.859 INFO:teuthology.orchestra.run.smithi007.stdout:(28/36): ceph-osd-19.0.0-2517.g95a4ad3c.el9.x86 3.9 MB/s | 17 MB 00:04 2024-05-08T15:44:35.884 INFO:teuthology.orchestra.run.smithi007.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 337 kB/s | 79 kB 00:00 2024-05-08T15:44:35.909 INFO:teuthology.orchestra.run.smithi007.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-05-08T15:44:35.953 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T15:44:35.953 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/4 2024-05-08T15:44:35.953 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/4 2024-05-08T15:44:35.953 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T15:44:36.009 INFO:teuthology.orchestra.run.smithi007.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 718 kB/s | 90 kB 00:00 2024-05-08T15:44:36.077 INFO:teuthology.orchestra.run.smithi007.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 257 kB/s | 36 kB 00:00 2024-05-08T15:44:36.135 INFO:teuthology.orchestra.run.smithi007.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-05-08T15:44:36.161 INFO:teuthology.orchestra.run.smithi007.stdout:(34/36): python3-portend-3.1.0-2.el9.noarch.rpm 54 kB/s | 16 kB 00:00 2024-05-08T15:44:36.228 INFO:teuthology.orchestra.run.smithi007.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.8 MB/s | 427 kB 00:00 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:36.238 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:44:36.253 INFO:teuthology.orchestra.run.smithi007.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 171 kB/s | 20 kB 00:00 2024-05-08T15:44:36.258 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:36.258 INFO:teuthology.orchestra.run.smithi007.stdout:Total 4.3 MB/s | 30 MB 00:07 2024-05-08T15:44:36.406 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:44:36.451 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:44:36.451 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:44:36.567 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph 2024-05-08T15:44:36.910 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:44:36.910 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:44:37.136 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:55 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:44:37.259 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 6.6 k 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 2.4 M 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 1.5 M 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 249 k 2024-05-08T15:44:37.262 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 4.7 M 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 17 M 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T15:44:37.263 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T15:44:37.264 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout:Install 36 Packages 2024-05-08T15:44:37.265 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:37.266 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 30 M 2024-05-08T15:44:37.266 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 107 M 2024-05-08T15:44:37.266 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:44:38.162 INFO:teuthology.orchestra.run.smithi081.stdout:(1/36): ceph-19.0.0-2517.g95a4ad3c.el9.x86_64.r 32 kB/s | 6.6 kB 00:00 2024-05-08T15:44:38.189 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:44:38.247 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T15:44:38.280 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T15:44:38.333 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T15:44:38.406 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T15:44:38.458 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T15:44:38.490 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T15:44:38.586 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T15:44:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T15:44:38.726 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T15:44:38.762 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T15:44:38.791 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T15:44:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T15:44:38.859 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T15:44:38.900 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T15:44:38.943 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T15:44:38.978 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T15:44:39.016 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T15:44:39.093 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T15:44:39.244 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T15:44:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T15:44:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T15:44:39.416 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T15:44:39.474 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T15:44:39.552 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T15:44:39.587 INFO:teuthology.orchestra.run.smithi081.stdout:(2/36): ceph-mgr-19.0.0-2517.g95a4ad3c.el9.x86_ 939 kB/s | 1.5 MB 00:01 2024-05-08T15:44:39.755 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T15:44:39.804 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T15:44:39.885 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T15:44:39.974 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T15:44:40.026 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T15:44:40.105 INFO:teuthology.orchestra.run.smithi081.stdout:(3/36): ceph-mds-19.0.0-2517.g95a4ad3c.el9.x86_ 1.1 MB/s | 2.4 MB 00:02 2024-05-08T15:44:40.153 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T15:44:40.179 INFO:teuthology.orchestra.run.smithi081.stdout:(4/36): ceph-mon-19.0.0-2517.g95a4ad3c.el9.x86_ 2.3 MB/s | 4.7 MB 00:02 2024-05-08T15:44:40.304 INFO:teuthology.orchestra.run.smithi081.stdout:(5/36): ceph-mgr-modules-core-19.0.0-2517.g95a4 1.2 MB/s | 249 kB 00:00 2024-05-08T15:44:40.325 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 31/36 2024-05-08T15:44:40.355 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T15:44:40.383 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T15:44:40.384 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:40.384 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T15:44:40.384 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T15:44:40.384 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T15:44:40.384 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:40.988 INFO:teuthology.orchestra.run.smithi081.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 313 kB/s | 253 kB 00:00 2024-05-08T15:44:41.305 INFO:teuthology.orchestra.run.smithi081.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 1.2 MB/s | 1.2 MB 00:01 2024-05-08T15:44:41.714 INFO:teuthology.orchestra.run.smithi081.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 147 kB/s | 106 kB 00:00 2024-05-08T15:44:41.906 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T15:44:41.935 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T15:44:41.935 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:41.935 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T15:44:41.935 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T15:44:41.935 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T15:44:41.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:42.055 INFO:teuthology.orchestra.run.smithi081.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 180 kB/s | 135 kB 00:00 2024-05-08T15:44:42.398 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T15:44:42.423 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T15:44:42.424 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:42.424 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T15:44:42.424 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T15:44:42.424 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T15:44:42.424 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:42.439 INFO:teuthology.orchestra.run.smithi081.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 172 kB/s | 125 kB 00:00 2024-05-08T15:44:42.572 INFO:teuthology.orchestra.run.smithi081.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-05-08T15:44:42.648 INFO:teuthology.orchestra.run.smithi081.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-08T15:44:42.657 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T15:44:42.683 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T15:44:42.683 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:42.683 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T15:44:42.683 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T15:44:42.683 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T15:44:42.683 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:42.690 INFO:teuthology.orchestra.run.smithi081.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 838 kB/s | 35 kB 00:00 2024-05-08T15:44:42.732 INFO:teuthology.orchestra.run.smithi081.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-08T15:44:42.740 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T15:44:42.774 INFO:teuthology.orchestra.run.smithi081.stdout:(15/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-08T15:44:42.841 INFO:teuthology.orchestra.run.smithi081.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 273 kB/s | 215 kB 00:00 2024-05-08T15:44:42.866 INFO:teuthology.orchestra.run.smithi081.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 243 kB/s | 22 kB 00:00 2024-05-08T15:44:43.241 INFO:teuthology.orchestra.run.smithi081.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 116 kB/s | 43 kB 00:00 2024-05-08T15:44:43.375 INFO:teuthology.orchestra.run.smithi081.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 283 kB/s | 151 kB 00:00 2024-05-08T15:44:43.517 INFO:teuthology.orchestra.run.smithi081.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 624 kB/s | 172 kB 00:00 2024-05-08T15:44:43.617 INFO:teuthology.orchestra.run.smithi081.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-05-08T15:44:43.717 INFO:teuthology.orchestra.run.smithi081.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 177 kB/s | 18 kB 00:00 2024-05-08T15:44:43.818 INFO:teuthology.orchestra.run.smithi081.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-05-08T15:44:43.843 INFO:teuthology.orchestra.run.smithi081.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 766 kB/s | 358 kB 00:00 2024-05-08T15:44:43.918 INFO:teuthology.orchestra.run.smithi081.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-05-08T15:44:43.944 INFO:teuthology.orchestra.run.smithi081.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-05-08T15:44:44.002 INFO:teuthology.orchestra.run.smithi081.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 553 kB/s | 46 kB 00:00 2024-05-08T15:44:44.036 INFO:teuthology.orchestra.run.smithi081.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 858 kB/s | 79 kB 00:00 2024-05-08T15:44:44.128 INFO:teuthology.orchestra.run.smithi081.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 179 kB/s | 16 kB 00:00 2024-05-08T15:44:44.220 INFO:teuthology.orchestra.run.smithi081.stdout:(30/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 980 kB/s | 90 kB 00:00 2024-05-08T15:44:44.320 INFO:teuthology.orchestra.run.smithi081.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 857 kB/s | 272 kB 00:00 2024-05-08T15:44:44.345 INFO:teuthology.orchestra.run.smithi081.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 285 kB/s | 36 kB 00:00 2024-05-08T15:44:44.504 INFO:teuthology.orchestra.run.smithi081.stdout:(33/36): ceph-osd-19.0.0-2517.g95a4ad3c.el9.x86 3.4 MB/s | 17 MB 00:04 2024-05-08T15:44:44.596 INFO:teuthology.orchestra.run.smithi081.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 833 kB/s | 230 kB 00:00 2024-05-08T15:44:44.650 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T15:44:44.650 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/36 2024-05-08T15:44:44.650 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/36 2024-05-08T15:44:44.650 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/36 2024-05-08T15:44:44.650 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/36 2024-05-08T15:44:44.650 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 6/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T15:44:44.651 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T15:44:44.652 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T15:44:44.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T15:44:44.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T15:44:44.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T15:44:44.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T15:44:44.653 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T15:44:44.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T15:44:44.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T15:44:44.654 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T15:44:44.692 INFO:teuthology.orchestra.run.smithi081.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.2 MB/s | 427 kB 00:00 2024-05-08T15:44:44.763 INFO:teuthology.orchestra.run.smithi081.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 77 kB/s | 20 kB 00:00 2024-05-08T15:44:44.766 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:44.766 INFO:teuthology.orchestra.run.smithi081.stdout:Total 4.0 MB/s | 30 MB 00:07 2024-05-08T15:44:44.909 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:44:44.949 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:44:44.950 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:44:45.416 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:44:45.417 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:44:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T15:44:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:45.655 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:44:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:45.655 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:45.656 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:44:45.656 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:45.656 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:45.656 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T15:44:45.656 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T15:44:45.657 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T15:44:45.657 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T15:44:45.657 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T15:44:45.657 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T15:44:45.657 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T15:44:45.657 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T15:44:45.658 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T15:44:45.658 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T15:44:45.658 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T15:44:45.658 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T15:44:45.658 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T15:44:45.658 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T15:44:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T15:44:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T15:44:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T15:44:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T15:44:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T15:44:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T15:44:45.659 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T15:44:45.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T15:44:45.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T15:44:45.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T15:44:45.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T15:44:45.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T15:44:45.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T15:44:45.660 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T15:44:45.661 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T15:44:45.661 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T15:44:45.661 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:45.661 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:45.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-base 2024-05-08T15:44:46.579 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:06 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:44:46.662 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:44:46.663 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:44:46.707 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:44:46.708 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:44:46.708 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:46.733 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T15:44:46.765 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T15:44:46.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install cephadm 2024-05-08T15:44:46.793 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T15:44:46.865 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T15:44:46.901 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T15:44:46.933 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T15:44:47.025 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T15:44:47.103 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T15:44:47.145 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T15:44:47.180 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T15:44:47.209 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T15:44:47.245 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T15:44:47.277 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T15:44:47.327 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T15:44:47.361 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T15:44:47.363 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:07 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:44:47.396 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T15:44:47.426 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T15:44:47.492 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:44:47.493 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:47.493 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:44:47.493 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:47.493 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 763 k 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 763 k 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 769 k 2024-05-08T15:44:47.494 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:44:47.499 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T15:44:47.640 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T15:44:47.718 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T15:44:47.770 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T15:44:47.809 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T15:44:47.895 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T15:44:47.970 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T15:44:48.033 INFO:teuthology.orchestra.run.smithi007.stdout:cephadm-19.0.0-2517.g95a4ad3c.el9.noarch.rpm 1.4 MB/s | 763 kB 00:00 2024-05-08T15:44:48.034 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:48.034 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.4 MB/s | 763 kB 00:00 2024-05-08T15:44:48.034 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:44:48.040 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:44:48.040 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:44:48.046 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:44:48.047 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:44:48.123 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:44:48.168 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T15:44:48.221 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T15:44:48.300 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T15:44:48.413 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T15:44:48.454 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T15:44:48.574 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T15:44:48.617 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:48.717 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:48.743 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 31/36 2024-05-08T15:44:48.773 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T15:44:48.799 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T15:44:48.799 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:48.799 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T15:44:48.799 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T15:44:48.799 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T15:44:48.799 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:49.055 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:49.306 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:44:49.306 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:44:49.307 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:49.307 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:49.524 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T15:44:50.116 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:10 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:44:50.240 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T15:44:50.244 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 145 k 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:44:50.245 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:50.246 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-05-08T15:44:50.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:50.246 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 145 k 2024-05-08T15:44:50.246 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 436 k 2024-05-08T15:44:50.246 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:44:50.268 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T15:44:50.268 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:50.268 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T15:44:50.268 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T15:44:50.268 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T15:44:50.268 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:50.600 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-immutable-object-cache-19.0.0-2517.g95a4ad 410 kB/s | 145 kB 00:00 2024-05-08T15:44:50.600 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:50.600 INFO:teuthology.orchestra.run.smithi007.stdout:Total 408 kB/s | 145 kB 00:00 2024-05-08T15:44:50.600 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:44:50.609 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:44:50.609 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:44:50.670 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:44:50.670 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:44:50.708 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T15:44:50.732 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T15:44:50.732 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:50.732 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T15:44:50.732 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T15:44:50.732 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T15:44:50.732 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:50.834 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:44:50.909 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T15:44:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T15:44:50.935 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:50.935 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T15:44:50.935 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:50.958 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T15:44:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T15:44:50.982 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:50.982 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T15:44:50.982 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T15:44:50.982 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T15:44:50.982 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:51.047 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T15:44:51.834 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T15:44:51.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:51.835 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:44:51.835 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:51.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:51.835 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:52.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr 2024-05-08T15:44:52.653 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:12 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:44:52.738 INFO:teuthology.orchestra.run.smithi007.stdout:Package ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:44:52.782 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:44:52.783 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:44:52.783 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/36 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/36 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/36 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/36 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/36 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 6/36 2024-05-08T15:44:52.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T15:44:52.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T15:44:52.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T15:44:52.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T15:44:52.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T15:44:52.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T15:44:52.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T15:44:52.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T15:44:52.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T15:44:53.429 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:13 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:44:53.557 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 3.5 M 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 32 k 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 16 k 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T15:44:53.558 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-05-08T15:44:53.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:53.560 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 8.6 M 2024-05-08T15:44:53.560 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 103 M 2024-05-08T15:44:53.560 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:53.885 INFO:teuthology.orchestra.run.smithi081.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T15:44:53.886 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T15:44:53.887 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T15:44:53.888 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T15:44:53.888 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:53.888 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:44:54.255 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-base 2024-05-08T15:44:54.270 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2517.g95a 65 kB/s | 16 kB 00:00 2024-05-08T15:44:54.295 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2517.g95 114 kB/s | 32 kB 00:00 2024-05-08T15:44:54.370 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 3.5 MB/s | 269 kB 00:00 2024-05-08T15:44:54.412 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-05-08T15:44:54.488 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 7.2 MB/s | 863 kB 00:00 2024-05-08T15:44:54.638 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 157 kB/s | 19 kB 00:00 2024-05-08T15:44:54.705 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.8 MB/s | 548 kB 00:00 2024-05-08T15:44:54.763 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): ceph-mgr-dashboard-19.0.0-2517.g95a4ad3 4.7 MB/s | 3.5 MB 00:00 2024-05-08T15:44:54.789 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.7 MB/s | 144 kB 00:00 2024-05-08T15:44:54.823 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:12 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:44:54.864 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.8 MB/s | 188 kB 00:00 2024-05-08T15:44:54.897 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 230 kB/s | 31 kB 00:00 2024-05-08T15:44:54.903 INFO:teuthology.orchestra.run.smithi081.stdout:Package ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:44:54.945 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:44:54.945 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:44:54.946 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:44:54.947 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.6 MB/s | 2.0 MB 00:00 2024-05-08T15:44:54.952 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:54.952 INFO:teuthology.orchestra.run.smithi007.stdout:Total 6.2 MB/s | 8.6 MB 00:01 2024-05-08T15:44:55.001 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:44:55.020 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:44:55.020 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:44:55.024 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install cephadm 2024-05-08T15:44:55.212 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:44:55.212 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:44:55.583 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:13 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:44:55.704 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 763 k 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 763 k 2024-05-08T15:44:55.705 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 769 k 2024-05-08T15:44:55.706 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:44:55.814 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:44:55.899 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T15:44:55.934 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T15:44:55.981 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T15:44:56.010 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T15:44:56.026 INFO:teuthology.orchestra.run.smithi081.stdout:cephadm-19.0.0-2517.g95a4ad3c.el9.noarch.rpm 2.3 MB/s | 763 kB 00:00 2024-05-08T15:44:56.026 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:56.026 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.3 MB/s | 763 kB 00:00 2024-05-08T15:44:56.026 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:44:56.032 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:44:56.032 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:44:56.037 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:44:56.038 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:44:56.107 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:44:56.123 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T15:44:56.260 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T15:44:56.359 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T15:44:56.417 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T15:44:56.486 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T15:44:56.517 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 10/12 2024-05-08T15:44:56.560 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:56.654 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:57.036 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:57.287 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T15:44:57.287 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:57.287 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:44:57.287 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:44:57.287 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:57.287 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:44:57.487 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T15:44:57.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 11/12 2024-05-08T15:44:57.651 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T15:44:58.060 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:16 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:44:58.183 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:44:58.184 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:58.184 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-05-08T15:44:58.184 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:58.184 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:44:58.184 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 145 k 2024-05-08T15:44:58.184 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:58.185 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:44:58.185 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:44:58.185 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-05-08T15:44:58.185 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:58.185 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 145 k 2024-05-08T15:44:58.185 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 436 k 2024-05-08T15:44:58.185 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:44:58.500 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-immutable-object-cache-19.0.0-2517.g95a4ad 460 kB/s | 145 kB 00:00 2024-05-08T15:44:58.501 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:44:58.501 INFO:teuthology.orchestra.run.smithi081.stdout:Total 458 kB/s | 145 kB 00:00 2024-05-08T15:44:58.501 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:44:58.508 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:44:58.508 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:44:58.564 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:44:58.564 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:44:58.711 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:44:58.800 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T15:44:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T15:44:58.826 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:44:58.826 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T15:44:58.826 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 1/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 2/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 3/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T15:44:59.057 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T15:44:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T15:44:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T15:44:59.058 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T15:44:59.361 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T15:44:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T15:44:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T15:44:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:44:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:44:59.612 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T15:44:59.705 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T15:44:59.706 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:59.706 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:44:59.706 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:44:59.706 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:44:59.706 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:44:59.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr 2024-05-08T15:45:00.200 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:20 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:00.328 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:00.329 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 7.4 M 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T15:45:00.330 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout:Install 12 Packages 2024-05-08T15:45:00.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:00.332 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 38 M 2024-05-08T15:45:00.332 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 209 M 2024-05-08T15:45:00.332 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:00.510 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:18 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:45:00.594 INFO:teuthology.orchestra.run.smithi081.stdout:Package ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:00.636 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:45:00.637 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:45:00.637 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:45:00.705 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T15:45:01.262 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:19 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:45:01.384 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:45:01.385 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 3.5 M 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 32 k 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 16 k 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T15:45:01.386 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout:Install 12 Packages 2024-05-08T15:45:01.387 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:01.388 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 8.6 M 2024-05-08T15:45:01.388 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 103 M 2024-05-08T15:45:01.388 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:45:01.473 INFO:teuthology.orchestra.run.smithi007.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 238 kB/s | 195 kB 00:00 2024-05-08T15:45:01.640 INFO:teuthology.orchestra.run.smithi007.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 815 kB/s | 803 kB 00:00 2024-05-08T15:45:01.664 INFO:teuthology.orchestra.run.smithi007.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 171 kB/s | 33 kB 00:00 2024-05-08T15:45:01.748 INFO:teuthology.orchestra.run.smithi007.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-05-08T15:45:01.832 INFO:teuthology.orchestra.run.smithi007.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-05-08T15:45:02.149 INFO:teuthology.orchestra.run.smithi007.stdout:(6/12): ceph-mgr-diskprediction-local-19.0.0-25 5.0 MB/s | 7.4 MB 00:01 2024-05-08T15:45:02.239 INFO:teuthology.orchestra.run.smithi081.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2517.g95a 54 kB/s | 16 kB 00:00 2024-05-08T15:45:02.306 INFO:teuthology.orchestra.run.smithi081.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2517.g95 86 kB/s | 32 kB 00:00 2024-05-08T15:45:02.399 INFO:teuthology.orchestra.run.smithi007.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 977 kB/s | 244 kB 00:00 2024-05-08T15:45:02.457 INFO:teuthology.orchestra.run.smithi007.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.7 MB/s | 3.0 MB 00:00 2024-05-08T15:45:02.525 INFO:teuthology.orchestra.run.smithi007.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.7 MB/s | 250 kB 00:00 2024-05-08T15:45:02.691 INFO:teuthology.orchestra.run.smithi007.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-08T15:45:02.723 INFO:teuthology.orchestra.run.smithi081.stdout:(3/12): ceph-mgr-dashboard-19.0.0-2517.g95a4ad3 4.5 MB/s | 3.5 MB 00:00 2024-05-08T15:45:03.040 INFO:teuthology.orchestra.run.smithi081.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.7 MB/s | 863 kB 00:00 2024-05-08T15:45:03.125 INFO:teuthology.orchestra.run.smithi007.stdout:(11/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 3.7 MB/s | 4.7 MB 00:01 2024-05-08T15:45:03.157 INFO:teuthology.orchestra.run.smithi081.stdout:(5/12): python3-protobuf-3.14.0-13.el9.noarch.r 316 kB/s | 269 kB 00:00 2024-05-08T15:45:03.232 INFO:teuthology.orchestra.run.smithi081.stdout:(6/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-05-08T15:45:03.333 INFO:teuthology.orchestra.run.smithi081.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 111 kB/s | 19 kB 00:00 2024-05-08T15:45:03.358 INFO:teuthology.orchestra.run.smithi081.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.7 MB/s | 548 kB 00:00 2024-05-08T15:45:03.392 INFO:teuthology.orchestra.run.smithi081.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 925 kB/s | 31 kB 00:00 2024-05-08T15:45:03.433 INFO:teuthology.orchestra.run.smithi081.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 1.4 MB/s | 144 kB 00:00 2024-05-08T15:45:03.476 INFO:teuthology.orchestra.run.smithi081.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-05-08T15:45:03.825 INFO:teuthology.orchestra.run.smithi007.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 12 MB/s | 16 MB 00:01 2024-05-08T15:45:03.826 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:03.826 INFO:teuthology.orchestra.run.smithi007.stdout:Total 11 MB/s | 38 MB 00:03 2024-05-08T15:45:04.017 INFO:teuthology.orchestra.run.smithi081.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.6 MB/s | 2.0 MB 00:00 2024-05-08T15:45:04.022 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:04.022 INFO:teuthology.orchestra.run.smithi081.stdout:Total 3.3 MB/s | 8.6 MB 00:02 2024-05-08T15:45:04.070 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:45:04.087 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:45:04.087 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:45:04.195 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:04.223 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:04.223 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:04.260 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:45:04.261 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:45:04.672 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:04.673 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:04.887 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:45:04.960 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T15:45:04.995 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T15:45:05.042 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T15:45:05.072 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T15:45:05.175 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T15:45:05.300 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T15:45:05.393 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T15:45:05.436 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T15:45:05.463 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:05.464 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T15:45:05.496 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 10/12 2024-05-08T15:45:05.529 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T15:45:05.574 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T15:45:05.678 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T15:45:05.725 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T15:45:05.909 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T15:45:05.940 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T15:45:06.019 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T15:45:06.547 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 11/12 2024-05-08T15:45:06.580 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T15:45:06.734 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T15:45:06.850 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 1/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 2/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 3/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T15:45:07.919 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T15:45:08.196 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T15:45:08.226 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T15:45:08.226 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:08.226 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:45:08.226 INFO:teuthology.orchestra.run.smithi081.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T15:45:08.226 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:08.226 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:08.226 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:08.227 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:45:08.494 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T15:45:09.063 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:01:27 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:45:09.185 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:45:09.186 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-05-08T15:45:09.186 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:45:09.186 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 7.4 M 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T15:45:09.187 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T15:45:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T15:45:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:09.188 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:45:09.188 INFO:teuthology.orchestra.run.smithi081.stdout:======================================================================================= 2024-05-08T15:45:09.188 INFO:teuthology.orchestra.run.smithi081.stdout:Install 12 Packages 2024-05-08T15:45:09.188 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:09.188 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 38 M 2024-05-08T15:45:09.189 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 209 M 2024-05-08T15:45:09.189 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:45:09.288 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T15:45:09.314 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T15:45:09.460 INFO:teuthology.orchestra.run.smithi081.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.2 MB/s | 195 kB 00:00 2024-05-08T15:45:09.585 INFO:teuthology.orchestra.run.smithi081.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.7 MB/s | 803 kB 00:00 2024-05-08T15:45:09.810 INFO:teuthology.orchestra.run.smithi081.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 100 kB/s | 33 kB 00:00 2024-05-08T15:45:09.919 INFO:teuthology.orchestra.run.smithi081.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 168 kB/s | 18 kB 00:00 2024-05-08T15:45:10.044 INFO:teuthology.orchestra.run.smithi081.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 282 kB/s | 35 kB 00:00 2024-05-08T15:45:10.670 INFO:teuthology.orchestra.run.smithi081.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-05-08T15:45:10.787 INFO:teuthology.orchestra.run.smithi081.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-05-08T15:45:11.120 INFO:teuthology.orchestra.run.smithi081.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 4.4 MB/s | 4.7 MB 00:01 2024-05-08T15:45:11.479 INFO:teuthology.orchestra.run.smithi081.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 699 kB/s | 250 kB 00:00 2024-05-08T15:45:11.935 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T15:45:11.935 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 1/12 2024-05-08T15:45:11.935 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T15:45:11.936 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T15:45:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T15:45:12.308 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T15:45:12.309 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:12.310 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:12.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-rook 2024-05-08T15:45:12.929 INFO:teuthology.orchestra.run.smithi081.stdout:(10/12): ceph-mgr-diskprediction-local-19.0.0-2 2.1 MB/s | 7.4 MB 00:03 2024-05-08T15:45:13.286 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:33 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:13.415 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 49 k 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T15:45:13.417 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout:Install 13 Packages 2024-05-08T15:45:13.418 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:13.419 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 2.2 M 2024-05-08T15:45:13.419 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 26 M 2024-05-08T15:45:13.419 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:13.955 INFO:teuthology.orchestra.run.smithi007.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 97 kB/s | 26 kB 00:00 2024-05-08T15:45:13.972 INFO:teuthology.orchestra.run.smithi007.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 68 kB/s | 19 kB 00:00 2024-05-08T15:45:13.989 INFO:teuthology.orchestra.run.smithi007.stdout:(3/13): ceph-mgr-rook-19.0.0-2517.g95a4ad3c.el9 166 kB/s | 49 kB 00:00 2024-05-08T15:45:14.339 INFO:teuthology.orchestra.run.smithi007.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 433 kB/s | 159 kB 00:00 2024-05-08T15:45:14.414 INFO:teuthology.orchestra.run.smithi007.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 484 kB/s | 222 kB 00:00 2024-05-08T15:45:14.449 INFO:teuthology.orchestra.run.smithi007.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 493 kB/s | 54 kB 00:00 2024-05-08T15:45:14.582 INFO:teuthology.orchestra.run.smithi007.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 470 kB/s | 279 kB 00:00 2024-05-08T15:45:14.757 INFO:teuthology.orchestra.run.smithi007.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 94 kB/s | 32 kB 00:00 2024-05-08T15:45:14.782 INFO:teuthology.orchestra.run.smithi007.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 45 kB/s | 15 kB 00:00 2024-05-08T15:45:14.983 INFO:teuthology.orchestra.run.smithi007.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 295 kB/s | 59 kB 00:00 2024-05-08T15:45:15.124 INFO:teuthology.orchestra.run.smithi007.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 380 kB/s | 206 kB 00:00 2024-05-08T15:45:15.150 INFO:teuthology.orchestra.run.smithi007.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 538 kB/s | 90 kB 00:00 2024-05-08T15:45:15.216 INFO:teuthology.orchestra.run.smithi007.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.2 MB/s | 1.0 MB 00:00 2024-05-08T15:45:15.220 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:15.220 INFO:teuthology.orchestra.run.smithi007.stdout:Total 1.2 MB/s | 2.2 MB 00:01 2024-05-08T15:45:15.294 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:15.312 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:15.405 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:15.407 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:15.654 INFO:teuthology.orchestra.run.smithi081.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 1.0 MB/s | 5.1 MB 00:04 2024-05-08T15:45:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:16.229 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T15:45:16.294 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T15:45:16.350 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T15:45:16.387 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T15:45:16.428 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T15:45:16.533 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T15:45:16.643 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T15:45:16.705 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T15:45:17.458 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T15:45:17.501 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T15:45:17.535 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T15:45:17.584 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T15:45:17.612 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T15:45:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T15:45:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/13 2024-05-08T15:45:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T15:45:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T15:45:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T15:45:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T15:45:18.482 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T15:45:18.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T15:45:18.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T15:45:18.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T15:45:18.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T15:45:18.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T15:45:18.483 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T15:45:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T15:45:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T15:45:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T15:45:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:18.816 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:19.057 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T15:45:19.648 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:39 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:19.776 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:19.777 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:45:19.777 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:19.777 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:19.777 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 142 k 2024-05-08T15:45:19.777 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout:Install 6 Packages 2024-05-08T15:45:19.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:19.779 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 7.0 M 2024-05-08T15:45:19.779 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 33 M 2024-05-08T15:45:19.779 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:20.127 INFO:teuthology.orchestra.run.smithi007.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-05-08T15:45:20.269 INFO:teuthology.orchestra.run.smithi007.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-08T15:45:20.485 INFO:teuthology.orchestra.run.smithi007.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.5 MB/s | 548 kB 00:00 2024-05-08T15:45:20.511 INFO:teuthology.orchestra.run.smithi007.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 240 kB/s | 58 kB 00:00 2024-05-08T15:45:20.561 INFO:teuthology.orchestra.run.smithi007.stdout:(5/6): ceph-mgr-cephadm-19.0.0-2517.g95a4ad3c.e 212 kB/s | 142 kB 00:00 2024-05-08T15:45:20.586 INFO:teuthology.orchestra.run.smithi007.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 677 kB/s | 51 kB 00:00 2024-05-08T15:45:20.589 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:20.589 INFO:teuthology.orchestra.run.smithi007.stdout:Total 8.6 MB/s | 7.0 MB 00:00 2024-05-08T15:45:20.662 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:20.675 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:20.675 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:20.778 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:20.779 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:21.086 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:21.268 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T15:45:21.331 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T15:45:21.889 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T15:45:21.977 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T15:45:22.047 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T15:45:22.075 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T15:45:22.872 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T15:45:22.872 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/6 2024-05-08T15:45:22.872 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T15:45:22.872 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T15:45:22.872 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T15:45:22.872 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T15:45:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T15:45:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:23.139 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T15:45:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T15:45:23.139 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T15:45:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T15:45:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T15:45:23.140 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:23.140 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:23.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-fuse 2024-05-08T15:45:23.954 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:43 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:24.084 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:24.084 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:24.084 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:45:24.084 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:24.084 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:24.084 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 890 k 2024-05-08T15:45:24.084 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:45:24.085 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T15:45:24.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:24.085 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:24.085 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:24.085 INFO:teuthology.orchestra.run.smithi007.stdout:Install 2 Packages 2024-05-08T15:45:24.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:24.085 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 970 k 2024-05-08T15:45:24.086 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 2.8 M 2024-05-08T15:45:24.086 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:24.729 INFO:teuthology.orchestra.run.smithi007.stdout:(1/2): ceph-fuse-19.0.0-2517.g95a4ad3c.el9.x86_ 2.3 MB/s | 890 kB 00:00 2024-05-08T15:45:25.105 INFO:teuthology.orchestra.run.smithi007.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 107 kB/s | 80 kB 00:00 2024-05-08T15:45:25.106 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:25.106 INFO:teuthology.orchestra.run.smithi007.stdout:Total 950 kB/s | 970 kB 00:01 2024-05-08T15:45:25.119 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:25.144 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:25.144 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:25.230 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:25.231 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:25.425 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:25.552 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T15:45:25.624 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T15:45:26.269 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T15:45:26.270 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/2 2024-05-08T15:45:26.561 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T15:45:26.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:26.561 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:26.561 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T15:45:26.561 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:26.561 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:26.869 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-volume 2024-05-08T15:45:27.469 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:27.596 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:27.596 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:27.596 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-05-08T15:45:27.596 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:27.596 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 264 k 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout:Install 2 Packages 2024-05-08T15:45:27.597 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:27.598 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 341 k 2024-05-08T15:45:27.598 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 1.5 M 2024-05-08T15:45:27.598 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:27.834 INFO:teuthology.orchestra.run.smithi007.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 458 kB/s | 77 kB 00:00 2024-05-08T15:45:28.184 INFO:teuthology.orchestra.run.smithi007.stdout:(2/2): ceph-volume-19.0.0-2517.g95a4ad3c.el9.no 508 kB/s | 264 kB 00:00 2024-05-08T15:45:28.185 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:28.185 INFO:teuthology.orchestra.run.smithi007.stdout:Total 579 kB/s | 341 kB 00:00 2024-05-08T15:45:28.199 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:28.209 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:28.210 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:28.245 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:28.245 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:28.513 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T15:45:28.729 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T15:45:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T15:45:28.746 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:45:28.746 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T15:45:28.746 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:29.419 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/2 2024-05-08T15:45:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T15:45:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:29.747 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:29.747 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T15:45:29.748 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:29.748 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:30.060 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados-devel 2024-05-08T15:45:30.655 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:30.781 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 127 k 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:30.782 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 127 k 2024-05-08T15:45:30.783 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 456 k 2024-05-08T15:45:30.783 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:31.002 INFO:teuthology.orchestra.run.smithi007.stdout:librados-devel-19.0.0-2517.g95a4ad3c.el9.x86_64 579 kB/s | 127 kB 00:00 2024-05-08T15:45:31.002 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:31.002 INFO:teuthology.orchestra.run.smithi007.stdout:Total 575 kB/s | 127 kB 00:00 2024-05-08T15:45:31.003 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:31.009 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:31.009 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:31.076 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:31.076 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:31.237 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:31.327 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:31.779 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:32.038 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:32.038 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:32.038 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:32.038 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:45:32.038 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:32.038 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:32.343 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs2 2024-05-08T15:45:32.950 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:33.038 INFO:teuthology.orchestra.run.smithi007.stdout:Package libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:33.082 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:33.083 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:45:33.083 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:33.151 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install libcephfs-devel 2024-05-08T15:45:33.738 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:53 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:33.865 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 32 k 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-05-08T15:45:33.866 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:33.867 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 32 k 2024-05-08T15:45:33.867 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 142 k 2024-05-08T15:45:33.867 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:34.119 INFO:teuthology.orchestra.run.smithi081.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 735 kB/s | 16 MB 00:22 2024-05-08T15:45:34.120 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:34.121 INFO:teuthology.orchestra.run.smithi081.stdout:Total 1.5 MB/s | 38 MB 00:24 2024-05-08T15:45:34.390 INFO:teuthology.orchestra.run.smithi007.stdout:libcephfs-devel-19.0.0-2517.g95a4ad3c.el9.x86_6 61 kB/s | 32 kB 00:00 2024-05-08T15:45:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:Total 60 kB/s | 32 kB 00:00 2024-05-08T15:45:34.391 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:34.395 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:34.409 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:34.410 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:34.477 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:45:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:34.503 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:45:34.503 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:45:34.618 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:34.933 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:45:34.933 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:45:34.980 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:35.202 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:35.202 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:35.202 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:35.202 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:45:35.203 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:35.203 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:35.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librados2 2024-05-08T15:45:35.689 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:45:35.753 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T15:45:35.793 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T15:45:35.901 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T15:45:35.944 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T15:45:35.962 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:55 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:36.047 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:36.091 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:36.092 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:45:36.092 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:36.122 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T15:45:36.151 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T15:45:36.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install librbd1 2024-05-08T15:45:36.228 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T15:45:36.763 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:56 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:36.848 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:36.893 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:36.894 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:45:36.894 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:36.896 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T15:45:36.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rados 2024-05-08T15:45:37.024 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T15:45:37.562 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:57 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:37.646 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:37.690 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:37.691 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:45:37.691 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:37.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rgw 2024-05-08T15:45:38.285 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T15:45:38.361 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:58 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:38.445 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:38.488 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:38.490 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:45:38.490 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:38.559 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-cephfs 2024-05-08T15:45:39.148 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:59 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:39.231 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:39.275 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:39.276 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:45:39.276 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:39.332 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T15:45:39.337 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install python3-rbd 2024-05-08T15:45:39.360 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T15:45:39.934 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:01:59 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:40.018 INFO:teuthology.orchestra.run.smithi007.stdout:Package python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:45:40.061 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:40.062 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-05-08T15:45:40.063 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:40.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-fuse 2024-05-08T15:45:40.831 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:02:00 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 85 k 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:40.959 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:40.960 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:40.960 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-05-08T15:45:40.960 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:40.960 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 85 k 2024-05-08T15:45:40.960 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 231 k 2024-05-08T15:45:40.960 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:41.410 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-fuse-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 189 kB/s | 85 kB 00:00 2024-05-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi007.stdout:Total 189 kB/s | 85 kB 00:00 2024-05-08T15:45:41.411 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:41.419 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:41.419 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:41.488 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:41.640 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:41.753 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:41.834 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T15:45:41.834 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 1/12 2024-05-08T15:45:41.834 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T15:45:41.834 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T15:45:41.834 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T15:45:41.834 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T15:45:41.835 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T15:45:41.835 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T15:45:41.835 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T15:45:41.835 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T15:45:41.835 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T15:45:41.835 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T15:45:42.151 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:42.207 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T15:45:42.207 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:42.207 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:45:42.207 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:42.207 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T15:45:42.207 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:42.208 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:45:42.412 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:42.413 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:42.413 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:42.413 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:45:42.413 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:42.413 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:42.577 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-rook 2024-05-08T15:45:42.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-mirror 2024-05-08T15:45:43.156 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:01 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:45:43.230 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:02:03 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:43.280 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:45:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:45:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:45:43.282 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 49 k 2024-05-08T15:45:43.282 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:43.283 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:45:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:Install 13 Packages 2024-05-08T15:45:43.284 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 2.2 M 2024-05-08T15:45:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 26 M 2024-05-08T15:45:43.284 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:45:43.358 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:43.359 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:43.359 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repo Size 2024-05-08T15:45:43.359 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:43.359 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 3.1 M 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 13 M 2024-05-08T15:45:43.360 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:43.628 INFO:teuthology.orchestra.run.smithi081.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 226 kB/s | 26 kB 00:00 2024-05-08T15:45:43.653 INFO:teuthology.orchestra.run.smithi081.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 139 kB/s | 19 kB 00:00 2024-05-08T15:45:43.736 INFO:teuthology.orchestra.run.smithi081.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-05-08T15:45:43.762 INFO:teuthology.orchestra.run.smithi081.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-05-08T15:45:43.804 INFO:teuthology.orchestra.run.smithi081.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 279 kB 00:00 2024-05-08T15:45:43.829 INFO:teuthology.orchestra.run.smithi081.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 800 kB/s | 54 kB 00:00 2024-05-08T15:45:43.871 INFO:teuthology.orchestra.run.smithi081.stdout:(7/13): ceph-mgr-rook-19.0.0-2517.g95a4ad3c.el9 138 kB/s | 49 kB 00:00 2024-05-08T15:45:44.063 INFO:teuthology.orchestra.run.smithi081.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 64 kB/s | 15 kB 00:00 2024-05-08T15:45:44.113 INFO:teuthology.orchestra.run.smithi081.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 104 kB/s | 32 kB 00:00 2024-05-08T15:45:44.197 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-mirror-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 3.7 MB/s | 3.1 MB 00:00 2024-05-08T15:45:44.198 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:44.198 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.7 MB/s | 3.1 MB 00:00 2024-05-08T15:45:44.199 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:44.210 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:44.210 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:44.257 INFO:teuthology.orchestra.run.smithi081.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 416 kB/s | 59 kB 00:00 2024-05-08T15:45:44.300 INFO:teuthology.orchestra.run.smithi081.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 483 kB/s | 206 kB 00:00 2024-05-08T15:45:44.301 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:44.301 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:44.405 INFO:teuthology.orchestra.run.smithi081.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.0 MB/s | 1.0 MB 00:00 2024-05-08T15:45:44.456 INFO:teuthology.orchestra.run.smithi081.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 448 kB/s | 90 kB 00:00 2024-05-08T15:45:44.459 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:44.459 INFO:teuthology.orchestra.run.smithi081.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-05-08T15:45:44.524 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:45:44.539 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:45:44.539 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:45:44.625 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:45:44.625 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:45:44.754 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:44.823 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:44.851 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:45:44.851 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T15:45:44.851 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T15:45:44.851 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T15:45:44.851 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:45.126 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:45:45.298 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T15:45:45.366 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T15:45:45.421 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T15:45:45.458 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T15:45:45.500 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T15:45:45.594 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T15:45:45.705 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T15:45:45.759 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T15:45:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:46.117 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:46.117 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:45:46.118 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:46.118 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:46.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install rbd-nbd 2024-05-08T15:45:46.490 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T15:45:46.532 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T15:45:46.574 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T15:45:46.622 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T15:45:46.652 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T15:45:46.961 INFO:teuthology.orchestra.run.smithi007.stdout:Last metadata expiration check: 0:02:06 ago on Wed 08 May 2024 03:43:40 PM UTC. 2024-05-08T15:45:47.089 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: Package Architecture Version Repository Size 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 170 k 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout:================================================================================ 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout:Install 1 Package 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:47.090 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 170 k 2024-05-08T15:45:47.091 INFO:teuthology.orchestra.run.smithi007.stdout:Installed size: 494 k 2024-05-08T15:45:47.091 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-05-08T15:45:47.411 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T15:45:47.412 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T15:45:47.561 INFO:teuthology.orchestra.run.smithi007.stdout:rbd-nbd-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 363 kB/s | 170 kB 00:00 2024-05-08T15:45:47.561 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:47.561 INFO:teuthology.orchestra.run.smithi007.stdout:Total 362 kB/s | 170 kB 00:00 2024-05-08T15:45:47.561 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-05-08T15:45:47.571 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-05-08T15:45:47.571 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-05-08T15:45:47.640 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-05-08T15:45:47.641 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-05-08T15:45:47.719 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T15:45:47.719 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:47.719 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:45:47.720 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:47.720 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T15:45:47.720 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T15:45:47.720 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T15:45:47.720 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T15:45:47.720 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T15:45:47.721 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T15:45:47.721 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T15:45:47.721 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T15:45:47.721 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T15:45:47.721 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T15:45:47.721 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T15:45:47.722 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T15:45:47.722 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:47.722 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:45:47.806 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-05-08T15:45:47.922 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:47.972 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T15:45:48.477 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:48.544 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:06 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:45:48.668 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:45:48.668 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:48.668 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:45:48.668 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 142 k 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:45:48.669 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:48.670 INFO:teuthology.orchestra.run.smithi081.stdout:Install 6 Packages 2024-05-08T15:45:48.670 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:48.670 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 7.0 M 2024-05-08T15:45:48.670 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 33 M 2024-05-08T15:45:48.670 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:45:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:45:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:48.734 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-05-08T15:45:48.734 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:45:48.735 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:45:48.735 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-05-08T15:45:48.982 DEBUG:teuthology.parallel:result is None 2024-05-08T15:45:49.293 INFO:teuthology.orchestra.run.smithi081.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2517.g95a4ad3c.e 373 kB/s | 142 kB 00:00 2024-05-08T15:45:49.568 INFO:teuthology.orchestra.run.smithi081.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 378 kB/s | 248 kB 00:00 2024-05-08T15:45:49.785 INFO:teuthology.orchestra.run.smithi081.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-05-08T15:45:49.810 INFO:teuthology.orchestra.run.smithi081.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 238 kB/s | 58 kB 00:00 2024-05-08T15:45:49.835 INFO:teuthology.orchestra.run.smithi081.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-05-08T15:45:53.860 INFO:teuthology.orchestra.run.smithi081.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.2 MB/s | 6.0 MB 00:04 2024-05-08T15:45:53.862 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:53.863 INFO:teuthology.orchestra.run.smithi081.stdout:Total 1.3 MB/s | 7.0 MB 00:05 2024-05-08T15:45:53.936 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:45:53.948 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:45:53.948 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:45:54.044 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:45:54.053 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:45:54.360 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:45:54.544 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T15:45:54.587 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T15:45:55.103 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T15:45:55.190 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T15:45:55.253 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T15:45:55.283 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T15:45:56.032 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T15:45:56.032 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/6 2024-05-08T15:45:56.032 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T15:45:56.032 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T15:45:56.032 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T15:45:56.032 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T15:45:56.299 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T15:45:56.299 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:56.300 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:45:56.542 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-fuse 2024-05-08T15:45:57.112 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:15 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:45:57.234 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 890 k 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:57.235 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:45:57.236 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:45:57.236 INFO:teuthology.orchestra.run.smithi081.stdout:Install 2 Packages 2024-05-08T15:45:57.236 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:57.236 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 970 k 2024-05-08T15:45:57.236 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 2.8 M 2024-05-08T15:45:57.236 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:45:57.530 INFO:teuthology.orchestra.run.smithi081.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 449 kB/s | 80 kB 00:00 2024-05-08T15:45:57.704 INFO:teuthology.orchestra.run.smithi081.stdout:(2/2): ceph-fuse-19.0.0-2517.g95a4ad3c.el9.x86_ 2.5 MB/s | 890 kB 00:00 2024-05-08T15:45:57.705 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:45:57.705 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.0 MB/s | 970 kB 00:00 2024-05-08T15:45:57.717 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:45:57.738 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:45:57.738 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:45:57.815 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:45:57.815 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:45:58.007 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:45:58.156 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T15:45:58.220 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T15:45:58.829 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T15:45:58.830 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/2 2024-05-08T15:45:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T15:45:59.091 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:59.091 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:45:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T15:45:59.092 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:45:59.092 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:45:59.374 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-volume 2024-05-08T15:45:59.953 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:17 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:00.086 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:00.086 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 264 k 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:Install 2 Packages 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 341 k 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 1.5 M 2024-05-08T15:46:00.087 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:46:00.502 INFO:teuthology.orchestra.run.smithi081.stdout:(1/2): ceph-volume-19.0.0-2517.g95a4ad3c.el9.no 1.1 MB/s | 264 kB 00:00 2024-05-08T15:46:00.685 INFO:teuthology.orchestra.run.smithi081.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 180 kB/s | 77 kB 00:00 2024-05-08T15:46:00.686 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:46:00.686 INFO:teuthology.orchestra.run.smithi081.stdout:Total 563 kB/s | 341 kB 00:00 2024-05-08T15:46:00.698 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:46:00.707 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:46:00.708 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:46:00.742 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:46:00.743 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:46:00.883 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:46:01.060 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T15:46:01.096 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T15:46:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T15:46:01.112 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:46:01.112 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T15:46:01.112 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:01.775 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/2 2024-05-08T15:46:02.049 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T15:46:02.049 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:02.049 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:46:02.050 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T15:46:02.050 INFO:teuthology.orchestra.run.smithi081.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T15:46:02.050 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:02.050 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:02.326 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librados-devel 2024-05-08T15:46:02.911 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:20 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:03.034 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:03.035 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:03.035 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-05-08T15:46:03.035 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:03.035 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:46:03.035 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 127 k 2024-05-08T15:46:03.035 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:03.035 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:46:03.036 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:03.036 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-05-08T15:46:03.036 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:03.036 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 127 k 2024-05-08T15:46:03.036 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 456 k 2024-05-08T15:46:03.036 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:46:03.584 INFO:teuthology.orchestra.run.smithi081.stdout:librados-devel-19.0.0-2517.g95a4ad3c.el9.x86_64 232 kB/s | 127 kB 00:00 2024-05-08T15:46:03.585 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:46:03.586 INFO:teuthology.orchestra.run.smithi081.stdout:Total 231 kB/s | 127 kB 00:00 2024-05-08T15:46:03.586 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:46:03.591 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:46:03.591 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:46:03.653 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:46:03.654 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:46:03.806 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:46:03.918 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:04.371 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:04.641 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:04.641 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:04.641 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:46:04.641 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:46:04.641 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:04.641 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:04.896 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs2 2024-05-08T15:46:05.474 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:23 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:05.556 INFO:teuthology.orchestra.run.smithi081.stdout:Package libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:46:05.599 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:05.600 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:46:05.600 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:05.662 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs-devel 2024-05-08T15:46:06.235 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:24 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:06.355 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 32 k 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:06.356 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 32 k 2024-05-08T15:46:06.357 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 142 k 2024-05-08T15:46:06.357 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:46:06.520 INFO:teuthology.orchestra.run.smithi081.stdout:libcephfs-devel-19.0.0-2517.g95a4ad3c.el9.x86_6 195 kB/s | 32 kB 00:00 2024-05-08T15:46:06.520 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:46:06.520 INFO:teuthology.orchestra.run.smithi081.stdout:Total 193 kB/s | 32 kB 00:00 2024-05-08T15:46:06.521 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:46:06.524 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:46:06.524 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:46:06.544 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:46:06.545 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:46:06.629 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:46:06.726 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:07.136 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:07.348 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:46:07.348 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:46:07.349 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:07.349 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:07.526 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librados2 2024-05-08T15:46:08.091 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:26 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:08.171 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:46:08.212 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:08.213 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:46:08.213 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:08.281 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install librbd1 2024-05-08T15:46:08.850 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:26 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:08.930 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:46:08.973 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:08.974 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:46:08.974 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:09.040 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rados 2024-05-08T15:46:09.619 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:27 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:09.705 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:46:09.749 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:09.750 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:46:09.750 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:09.823 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rgw 2024-05-08T15:46:10.425 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:28 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:10.507 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:46:10.550 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:10.551 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:46:10.551 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:10.624 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-cephfs 2024-05-08T15:46:11.190 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:29 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:11.270 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:46:11.311 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:11.312 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:46:11.313 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:11.386 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install python3-rbd 2024-05-08T15:46:11.959 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:29 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:12.041 INFO:teuthology.orchestra.run.smithi081.stdout:Package python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T15:46:12.082 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:12.083 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do. 2024-05-08T15:46:12.083 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:12.148 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-fuse 2024-05-08T15:46:12.718 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:30 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:12.840 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:12.841 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:12.841 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-05-08T15:46:12.841 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:12.841 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 85 k 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 85 k 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 231 k 2024-05-08T15:46:12.842 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:46:13.165 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-fuse-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 264 kB/s | 85 kB 00:00 2024-05-08T15:46:13.166 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:46:13.166 INFO:teuthology.orchestra.run.smithi081.stdout:Total 263 kB/s | 85 kB 00:00 2024-05-08T15:46:13.166 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:46:13.173 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:46:13.173 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:46:13.237 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:46:13.237 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:46:13.384 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:46:13.496 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:13.869 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:14.139 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:14.139 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:14.139 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:46:14.139 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:46:14.139 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:14.139 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:14.357 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-mirror 2024-05-08T15:46:14.942 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:32 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:15.068 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:15.068 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repo Size 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 3.1 M 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 13 M 2024-05-08T15:46:15.069 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:46:15.810 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-mirror-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 4.2 MB/s | 3.1 MB 00:00 2024-05-08T15:46:15.811 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:46:15.812 INFO:teuthology.orchestra.run.smithi081.stdout:Total 4.2 MB/s | 3.1 MB 00:00 2024-05-08T15:46:15.812 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:46:15.821 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:46:15.821 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:46:15.903 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:46:15.904 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:46:16.336 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:46:16.395 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:16.421 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:16.421 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T15:46:16.421 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T15:46:16.421 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T15:46:16.422 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T15:46:16.422 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:17.448 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:17.449 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:17.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-nbd 2024-05-08T15:46:18.250 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:02:36 ago on Wed 08 May 2024 03:43:42 PM UTC. 2024-05-08T15:46:18.377 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-05-08T15:46:18.378 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:18.378 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-05-08T15:46:18.378 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:18.378 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 170 k 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 170 k 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 494 k 2024-05-08T15:46:18.379 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-05-08T15:46:18.639 INFO:teuthology.orchestra.run.smithi081.stdout:rbd-nbd-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 656 kB/s | 170 kB 00:00 2024-05-08T15:46:18.640 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-05-08T15:46:18.640 INFO:teuthology.orchestra.run.smithi081.stdout:Total 652 kB/s | 170 kB 00:00 2024-05-08T15:46:18.640 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-05-08T15:46:18.648 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-05-08T15:46:18.648 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-05-08T15:46:18.714 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-05-08T15:46:18.714 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-05-08T15:46:18.869 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-05-08T15:46:18.963 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:19.408 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:19.639 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T15:46:19.639 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:19.639 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-05-08T15:46:19.639 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T15:46:19.639 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:19.639 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-05-08T15:46:19.917 DEBUG:teuthology.parallel:result is None 2024-05-08T15:46:19.918 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T15:46:20.058 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T15:46:20.085 INFO:teuthology.orchestra.run.smithi007.stdout:19.0.0-2517.g95a4ad3c.el9 2024-05-08T15:46:20.085 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2517.g95a4ad3c.el9 2024-05-08T15:46:20.085 INFO:teuthology.task.install:The correct ceph version 19.0.0-2517.g95a4ad3c is installed. 2024-05-08T15:46:20.087 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T15:46:20.299 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T15:46:20.321 INFO:teuthology.orchestra.run.smithi081.stdout:19.0.0-2517.g95a4ad3c.el9 2024-05-08T15:46:20.321 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2517.g95a4ad3c.el9 2024-05-08T15:46:20.321 INFO:teuthology.task.install:The correct ceph version 19.0.0-2517.g95a4ad3c is installed. 2024-05-08T15:46:20.323 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T15:46:20.323 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:20.323 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T15:46:20.356 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:20.356 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T15:46:20.386 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T15:46:20.386 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:20.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T15:46:20.426 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T15:46:20.496 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:20.496 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T15:46:20.520 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T15:46:20.585 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T15:46:20.585 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:20.585 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T15:46:20.615 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T15:46:20.683 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:20.683 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T15:46:20.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T15:46:20.773 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T15:46:20.773 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:20.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T15:46:20.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T15:46:20.865 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:20.865 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T15:46:20.890 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T15:46:20.954 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T15:46:21.034 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T15:46:21.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-05-08T15:46:21.037 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /var/log/ceph 2024-05-08T15:46:21.061 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T15:46:21.061 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T15:46:21.104 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T15:46:21.128 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T15:46:21.128 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T15:46:21.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T15:46:21.204 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T15:46:21.204 INFO:tasks.ceph:config {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'slow request', 'overall HEALTH_'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-05-08T15:46:21.204 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604', 'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid', 'description': 'rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete}', 'email': None, 'first_in_suite': False, 'job_id': '7697604', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid'}, 'ceph': {'conf': {'client': {'debug monc': 20, 'debug ms': 1}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'ScrubResult', 'scrub mismatch', 'overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_AVAILABILITY\\)', '\\(SLOW_OPS\\)', 'slow request', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'slow request', 'overall HEALTH_'], 'mon_bind_msgr2': False, 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}}, 'owner': 'scheduled_lflores@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mon.d', 'mon.e', 'osd.0', 'osd.1', 'osd.2'], ['mon.f', 'mon.g', 'mon.h', 'mon.i', 'mgr.x', 'osd.3', 'osd.4', 'osd.5', 'client.0']], 'seed': 8082, 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-lflores-testing-2-2024-05-07-1606-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBczkMex5Mf99nGnIe00Fz2HEdqIoPkZbBaLijHg9c3JfJMQs1aH+scqmEQNI4/ZZWo9ylBnK+JzYTCwy8VbF5M=', 'smithi081.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJFlyhAXJ2YIeYFuR/c5NEwAK94zifa9NMoYi8r2HVrS51RR0Z9byk2Kir2kWlPXPg8sj/61QiM+IolxhdGuhX8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'mon_thrash': {'revive_delay': 90, 'thrash_delay': 1, 'thrash_many': True}}, {'exec': {'client.0': ['ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel', 'ceph_test_rados_delete_pools_parallel']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_14:59:36', 'tube': 'smithi', 'user': 'lflores', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T15:46:21.205 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T15:46:21.236 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T15:46:21.252 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T15:46:21.293 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T15:46:21.318 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:21.318 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T15:46:21.373 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T15:46:21.374 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-08 15:45:45.126088666 +0000 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-08 15:42:58.644676277 +0000 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-08 15:42:58.644676277 +0000 2024-05-08T15:46:21.431 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-08 15:42:58.644676277 +0000 2024-05-08T15:46:21.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T15:46:21.496 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-08T15:46:21.496 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-08T15:46:21.496 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 8.1477e-05 s, 6.3 MB/s 2024-05-08T15:46:21.497 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T15:46:21.555 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-08 15:45:45.127088651 +0000 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-08 15:42:58.983671178 +0000 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-08 15:42:58.983671178 +0000 2024-05-08T15:46:21.611 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-08 15:42:58.983671178 +0000 2024-05-08T15:46:21.611 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T15:46:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-08T15:46:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-08T15:46:21.675 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 9.6436e-05 s, 5.3 MB/s 2024-05-08T15:46:21.676 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T15:46:21.733 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-05-08T15:46:21.789 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-08 15:45:45.127088651 +0000 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-08 15:42:59.326666019 +0000 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-08 15:42:59.326666019 +0000 2024-05-08T15:46:21.790 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-08 15:42:59.326666019 +0000 2024-05-08T15:46:21.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T15:46:21.855 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-08T15:46:21.855 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-08T15:46:21.855 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000102673 s, 5.0 MB/s 2024-05-08T15:46:21.856 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T15:46:21.914 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-05-08 15:45:45.128088636 +0000 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-05-08 15:42:59.665660920 +0000 2024-05-08T15:46:21.970 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-05-08 15:42:59.665660920 +0000 2024-05-08T15:46:21.971 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-05-08 15:42:59.665660920 +0000 2024-05-08T15:46:21.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T15:46:22.034 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-05-08T15:46:22.034 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-05-08T15:46:22.034 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 8.5271e-05 s, 6.0 MB/s 2024-05-08T15:46:22.035 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T15:46:22.092 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:22.092 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:22.092 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T15:46:22.110 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T15:46:22.111 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-05-08T15:46:22.168 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T15:46:22.168 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:22.168 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-08T15:46:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 15:46:16.663862703 +0000 2024-05-08T15:46:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 15:42:53.015600660 +0000 2024-05-08T15:46:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 15:42:53.015600660 +0000 2024-05-08T15:46:22.169 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-08 15:42:53.015600660 +0000 2024-05-08T15:46:22.169 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T15:46:22.234 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T15:46:22.234 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T15:46:22.234 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000198273 s, 2.6 MB/s 2024-05-08T15:46:22.235 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T15:46:22.293 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-05-08T15:46:22.350 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T15:46:22.350 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:22.350 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-08T15:46:22.350 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:22.350 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:22.350 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 15:46:16.664862684 +0000 2024-05-08T15:46:22.350 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 15:42:53.550590268 +0000 2024-05-08T15:46:22.351 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 15:42:53.550590268 +0000 2024-05-08T15:46:22.351 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-08 15:42:53.550590268 +0000 2024-05-08T15:46:22.351 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T15:46:22.415 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T15:46:22.415 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T15:46:22.415 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.00021779 s, 2.4 MB/s 2024-05-08T15:46:22.417 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T15:46:22.474 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-05-08T15:46:22.530 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 15:46:16.664862684 +0000 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 15:42:53.895583567 +0000 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 15:42:53.895583567 +0000 2024-05-08T15:46:22.531 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-08 15:42:53.895583567 +0000 2024-05-08T15:46:22.531 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T15:46:22.595 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T15:46:22.595 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T15:46:22.595 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000224202 s, 2.3 MB/s 2024-05-08T15:46:22.597 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T15:46:22.654 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-05-08T15:46:22.710 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-05-08 15:46:16.664862684 +0000 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-05-08 15:42:54.249576691 +0000 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-05-08 15:42:54.249576691 +0000 2024-05-08T15:46:22.711 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-05-08 15:42:54.249576691 +0000 2024-05-08T15:46:22.711 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T15:46:22.775 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-05-08T15:46:22.775 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-05-08T15:46:22.776 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000201193 s, 2.5 MB/s 2024-05-08T15:46:22.777 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T15:46:22.834 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:22.834 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-05-08T15:46:22.834 INFO:tasks.ceph:Generating config... 2024-05-08T15:46:22.835 INFO:tasks.ceph:[client] debug monc = 20 2024-05-08T15:46:22.836 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T15:46:22.836 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-08T15:46:22.836 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-05-08T15:46:22.836 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-05-08T15:46:22.836 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-08T15:46:22.836 INFO:tasks.ceph:[global] ms type = async 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mon] mon min osdmap epochs = 25 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-05-08T15:46:22.836 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-05-08T15:46:22.837 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-05-08T15:46:22.837 INFO:tasks.ceph:[mon] mon scrub inject crc mismatch = 0.01 2024-05-08T15:46:22.837 INFO:tasks.ceph:[mon] mon scrub inject missing keys = 0.05 2024-05-08T15:46:22.837 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-05-08T15:46:22.837 INFO:tasks.ceph:[mon] paxos min = 10 2024-05-08T15:46:22.837 INFO:tasks.ceph:[mon] paxos service trim min = 5 2024-05-08T15:46:22.837 INFO:tasks.ceph:[mon] paxos trim min = 10 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-08T15:46:22.837 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-05-08T15:46:22.838 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-05-08T15:46:22.838 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T15:46:22.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:22.899 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T15:46:22.902 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T15:46:22.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T15:46:22.976 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.7:6789]'), ('mon.b', '[v1:172.21.15.7:6790]'), ('mon.c', '[v1:172.21.15.7:6791]'), ('mon.d', '[v1:172.21.15.7:6792]'), ('mon.e', '[v1:172.21.15.7:6793]'), ('mon.f', '[v1:172.21.15.81:6789]'), ('mon.g', '[v1:172.21.15.81:6790]'), ('mon.h', '[v1:172.21.15.81:6791]'), ('mon.i', '[v1:172.21.15.81:6792]')] 2024-05-08T15:46:22.977 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.7:6789],[v1:172.21.15.7:6790],[v1:172.21.15.7:6791],[v1:172.21.15.7:6792],[v1:172.21.15.7:6793],[v1:172.21.15.81:6789],[v1:172.21.15.81:6790],[v1:172.21.15.81:6791],[v1:172.21.15.81:6792]', 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 25, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub inject crc mismatch': 0.01, 'mon scrub inject missing keys': 0.05, 'mon scrub interval': 300, 'paxos min': 10, 'paxos service trim min': 5, 'paxos trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug monc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mon.d': {}, 'mon.e': {}, 'mon.f': {}, 'mon.g': {}, 'mon.h': {}, 'mon.i': {}} 2024-05-08T15:46:22.977 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:22.978 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T15:46:23.033 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.7:6789]' --addv b '[v1:172.21.15.7:6790]' --addv c '[v1:172.21.15.7:6791]' --addv d '[v1:172.21.15.7:6792]' --addv e '[v1:172.21.15.7:6793]' --addv f '[v1:172.21.15.81:6789]' --addv g '[v1:172.21.15.81:6790]' --addv h '[v1:172.21.15.81:6791]' --addv i '[v1:172.21.15.81:6792]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:23.140 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 3 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:1: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:2: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:3: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:4: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:23.141 INFO:teuthology.orchestra.run.smithi007.stdout:5: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:23.142 INFO:teuthology.orchestra.run.smithi007.stdout:6: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:23.142 INFO:teuthology.orchestra.run.smithi007.stdout:7: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:23.142 INFO:teuthology.orchestra.run.smithi007.stdout:8: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:23.142 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (9 monitors) 2024-05-08T15:46:23.143 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T15:46:23.159 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 402a637b-159c-4d0b-9f9c-7ff72b154b33... 2024-05-08T15:46:23.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T15:46:23.202 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T15:46:23.243 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.244 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-05-08T15:46:23.245 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.246 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "[v1:172.21.15.7:6789],[v1:172.21.15.7:6790],[v1:172.21.15.7:6791],[v1:172.21.15.7:6792],[v1:172.21.15.7:6793],[v1:172.21.15.81:6789],[v1:172.21.15.81:6790],[v1:172.21.15.81:6791],[v1:172.21.15.81:6792]" 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: mon client directed command retry = 5 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: mon election default strategy = 3 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: ms bind msgr2 = False 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: ms inject socket failures = 5000 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: ms type = async 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-05-08T15:46:23.247 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-05-08T15:46:23.248 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore compression algorithm = zlib 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore compression mode = aggressive 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore zero block detection = True 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 20 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 20 2024-05-08T15:46:23.249 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 10 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify cached snaps = True 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify missing on start = True 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock override recovery settings = True 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: osd op queue = debug_random 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: osd op queue cut off = debug_random 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-05-08T15:46:23.250 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-05-08T15:46:23.251 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon min osdmap epochs = 25 2024-05-08T15:46:23.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon osdmap full prune interval = 2 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: mon osdmap full prune min = 15 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: mon osdmap full prune txsize = 2 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: mon scrub inject crc mismatch = 0.01 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: mon scrub inject missing keys = 0.05 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: mon scrub interval = 300 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: paxos min = 10 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: paxos service trim min = 5 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: paxos trim min = 10 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T15:46:23.253 INFO:teuthology.orchestra.run.smithi007.stdout: debug monc = 20 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.d] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.e] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.f] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.g] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.h] 2024-05-08T15:46:23.254 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.i] 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2024-05-08T15:46:23.255 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2024-05-08T15:46:23.256 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2024-05-08T15:46:23.257 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "[v1:172.21.15.7:6789],[v1:172.21.15.7:6790],[v1:172.21.15.7:6791],[v1:172.21.15.7:6792],[v1:172.21.15.7:6793],[v1:172.21.15.81:6789],[v1:172.21.15.81:6790],[v1:172.21.15.81:6791],[v1:172.21.15.81:6792]" 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: mon client directed command retry = 5 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: mon election default strategy = 3 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: ms bind msgr2 = False 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: ms inject socket failures = 5000 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: ms type = async 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2024-05-08T15:46:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore compression algorithm = zlib 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore compression mode = aggressive 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore zero block detection = True 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 20 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 20 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 10 2024-05-08T15:46:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify cached snaps = True 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify missing on start = True 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock override recovery settings = True 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: osd op queue = debug_random 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: osd op queue cut off = debug_random 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-05-08T15:46:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.262 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon min osdmap epochs = 25 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon osdmap full prune interval = 2 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon osdmap full prune min = 15 2024-05-08T15:46:23.263 INFO:teuthology.orchestra.run.smithi081.stdout: mon osdmap full prune txsize = 2 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: mon scrub inject crc mismatch = 0.01 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: mon scrub inject missing keys = 0.05 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: mon scrub interval = 300 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: paxos min = 10 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: paxos service trim min = 5 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: paxos trim min = 10 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: debug monc = 20 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2024-05-08T15:46:23.264 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.d] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.e] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.f] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.g] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.h] 2024-05-08T15:46:23.265 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.i] 2024-05-08T15:46:23.266 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T15:46:23.266 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T15:46:23.362 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T15:46:23.363 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:23.363 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T15:46:23.378 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:23.378 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T15:46:23.434 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:46:23.434 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:23.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T15:46:23.434 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T15:46:23.510 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:23.510 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:23.566 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:46:23.567 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:23.567 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T15:46:23.567 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T15:46:23.607 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:23.607 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:23.665 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T15:46:23.665 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T15:46:23.665 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T15:46:23.769 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T15:46:23.772 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T15:46:23.772 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T15:46:23.772 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T15:46:23.814 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T15:46:23.827 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T15:46:23.827 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'}} 2024-05-08T15:46:23.827 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T15:46:23.853 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:23.853 INFO:tasks.ceph:role: osd.0 2024-05-08T15:46:23.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:46:23.853 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T15:46:23.919 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T15:46:23.919 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T15:46:23.920 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T15:46:23.952 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-05-08T15:46:23.954 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-05-08T15:46:23.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T15:46:24.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T15:46:24.094 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T15:46:24.162 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:24.162 INFO:tasks.ceph:role: osd.1 2024-05-08T15:46:24.162 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:46:24.162 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T15:46:24.228 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T15:46:24.228 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T15:46:24.228 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T15:46:24.229 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T15:46:24.229 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T15:46:24.229 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-05-08T15:46:24.229 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T15:46:24.229 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T15:46:24.229 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T15:46:24.229 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T15:46:24.261 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-05-08T15:46:24.263 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-05-08T15:46:24.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T15:46:24.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T15:46:24.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T15:46:24.468 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:24.468 INFO:tasks.ceph:role: osd.2 2024-05-08T15:46:24.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:46:24.469 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T15:46:24.533 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T15:46:24.533 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T15:46:24.533 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T15:46:24.533 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T15:46:24.534 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T15:46:24.534 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-05-08T15:46:24.534 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T15:46:24.534 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T15:46:24.534 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T15:46:24.534 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T15:46:24.566 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-05-08T15:46:24.568 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-05-08T15:46:24.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T15:46:24.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T15:46:24.707 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:24.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:24.813+0000 7f5ecd611640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T15:46:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:24.813+0000 7f5ecd611640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T15:46:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:24.813+0000 7f5ecd611640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T15:46:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:24.813+0000 7f5ecd611640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T15:46:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:24.813+0000 7f5ecd611640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T15:46:24.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:24.814+0000 7f5ecd611640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T15:46:27.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T15:46:27.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:27.787+0000 7f45ef0d9640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T15:46:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:27.787+0000 7f45ef0d9640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T15:46:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:27.787+0000 7f45ef0d9640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T15:46:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:27.787+0000 7f45ef0d9640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T15:46:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:27.787+0000 7f45ef0d9640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T15:46:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:27.787+0000 7f45ef0d9640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T15:46:30.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T15:46:30.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:30.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:30.752+0000 7f1f569b8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T15:46:30.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:30.752+0000 7f1f569b8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T15:46:30.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:30.752+0000 7f1f569b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T15:46:30.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:30.752+0000 7f1f569b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T15:46:30.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:30.752+0000 7f1f569b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T15:46:30.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:30.753+0000 7f1f569b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T15:46:33.598 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T15:46:33.628 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T15:46:33.658 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:33.658 INFO:tasks.ceph:role: osd.3 2024-05-08T15:46:33.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:46:33.658 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T15:46:33.734 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T15:46:33.734 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T15:46:33.734 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T15:46:33.735 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T15:46:33.735 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T15:46:33.735 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-05-08T15:46:33.735 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T15:46:33.735 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T15:46:33.736 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T15:46:33.736 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T15:46:34.248 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-05-08T15:46:34.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-05-08T15:46:34.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-05-08T15:46:34.376 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T15:46:34.412 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T15:46:34.481 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:34.481 INFO:tasks.ceph:role: osd.4 2024-05-08T15:46:34.481 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:46:34.481 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T15:46:34.556 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T15:46:35.008 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-05-08T15:46:35.010 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-05-08T15:46:35.010 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-05-08T15:46:35.099 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T15:46:35.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T15:46:35.192 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.5': '/dev/vg_nvme/lv_3'} 2024-05-08T15:46:35.192 INFO:tasks.ceph:role: osd.5 2024-05-08T15:46:35.193 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:46:35.193 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T15:46:35.265 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T15:46:35.265 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T15:46:35.265 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T15:46:35.265 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T15:46:35.265 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T15:46:35.266 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2024-05-08T15:46:35.266 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T15:46:35.266 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T15:46:35.266 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T15:46:35.266 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T15:46:35.754 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2024-05-08T15:46:35.756 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2024-05-08T15:46:35.756 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-5 2024-05-08T15:46:35.905 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T15:46:35.934 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:36.026 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:36.030+0000 7fe021617640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T15:46:36.030 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:36.034+0000 7fe021617640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T15:46:36.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:36.034+0000 7fe021617640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T15:46:36.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:36.034+0000 7fe021617640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T15:46:36.031 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:36.034+0000 7fe021617640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T15:46:36.037 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:36.041+0000 7fe021617640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T15:46:38.888 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T15:46:38.917 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:39.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:39.010+0000 7f6c7380c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T15:46:39.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:39.011+0000 7f6c7380c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T15:46:39.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:39.011+0000 7f6c7380c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T15:46:39.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:39.011+0000 7f6c7380c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T15:46:39.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:39.011+0000 7f6c7380c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T15:46:39.008 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:39.011+0000 7f6c7380c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T15:46:41.858 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T15:46:41.888 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:41.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:41.982+0000 7f0dd3443640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T15:46:41.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:41.982+0000 7f0dd3443640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T15:46:41.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:41.982+0000 7f0dd3443640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T15:46:41.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:41.982+0000 7f0dd3443640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T15:46:41.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:41.982+0000 7f0dd3443640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T15:46:41.979 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:41.983+0000 7f0dd3443640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T15:46:44.837 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T15:46:44.866 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T15:46:44.867 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:44.867 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T15:46:44.899 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:44.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T15:46:44.969 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:44.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T15:46:45.038 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:45.038 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T15:46:45.068 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:45.068 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T15:46:45.137 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:45.137 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T15:46:45.206 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:45.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T15:46:45.273 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:45.274 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T15:46:45.331 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T15:46:45.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T15:46:45.334 DEBUG:teuthology.orchestra.run.smithi081:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T15:46:45.374 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBQnjtmFmKNMBAAHJc324ckmAHYu1KyVE/4YQ== 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBTnjtm33z4LhAAbeKwHpydArPpLW141WBL3w== 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBWnjtmVcHoLBAAr+bVfFI0gaPNtgejSAcm/Q== 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBPnjtmX5AmLhAA6xEk2sDNGKYTwLZbewQnqA== 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBcnjtmk/LcARAANmxH4330HXe5741SQ5dgIg== 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBfnjtm5qy2ABAAqYrKB/IxD4cgK2cD8R/1zA== 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-05-08T15:46:45.375 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBhnjtmnbWcOhAAKDZLhFLtrlELYACIK+0iag== 2024-05-08T15:46:45.376 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-05-08T15:46:45.376 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQBPnjtm3bXTMBAAuH0o789rYc+DJI0m1zaxNg== 2024-05-08T15:46:45.396 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.0] 2024-05-08T15:46:45.396 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBQnjtmFmKNMBAAHJc324ckmAHYu1KyVE/4YQ== 2024-05-08T15:46:45.396 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.1] 2024-05-08T15:46:45.396 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBTnjtm33z4LhAAbeKwHpydArPpLW141WBL3w== 2024-05-08T15:46:45.396 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.2] 2024-05-08T15:46:45.396 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBWnjtmVcHoLBAAr+bVfFI0gaPNtgejSAcm/Q== 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.x] 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBPnjtmX5AmLhAA6xEk2sDNGKYTwLZbewQnqA== 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.3] 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBcnjtmk/LcARAANmxH4330HXe5741SQ5dgIg== 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.4] 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBfnjtm5qy2ABAAqYrKB/IxD4cgK2cD8R/1zA== 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.5] 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBhnjtmnbWcOhAAKDZLhFLtrlELYACIK+0iag== 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2024-05-08T15:46:45.397 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBPnjtm3bXTMBAAuH0o789rYc+DJI0m1zaxNg== 2024-05-08T15:46:45.398 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.420 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.494 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.602 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T15:46:45.605 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T15:46:45.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.658 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.710 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T15:46:45.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T15:46:45.817 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T15:46:45.870 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T15:46:45.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T15:46:45.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:45.993 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T15:46:46.021 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T15:46:46.089 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:46.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T15:46:46.213 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T15:46:46.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:46.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T15:46:46.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-d 2024-05-08T15:46:46.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i d --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:46.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-d 2024-05-08T15:46:46.594 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-e 2024-05-08T15:46:46.660 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i e --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:46.757 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-e 2024-05-08T15:46:46.784 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-f 2024-05-08T15:46:46.813 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i f --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:46.914 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-f 2024-05-08T15:46:46.940 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-g 2024-05-08T15:46:47.006 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i g --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:47.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-g 2024-05-08T15:46:47.128 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-h 2024-05-08T15:46:47.193 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i h --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:47.288 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-h 2024-05-08T15:46:47.314 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-i 2024-05-08T15:46:47.379 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i i --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T15:46:47.473 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-i 2024-05-08T15:46:47.498 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:47.501 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T15:46:47.553 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T15:46:47.554 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T15:46:47.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T15:46:47.560 INFO:tasks.ceph.mon.a:Started 2024-05-08T15:46:47.560 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T15:46:47.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T15:46:47.562 INFO:tasks.ceph.mon.b:Started 2024-05-08T15:46:47.562 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T15:46:47.562 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T15:46:47.565 INFO:tasks.ceph.mon.c:Started 2024-05-08T15:46:47.565 INFO:tasks.ceph.mon.d:Restarting daemon 2024-05-08T15:46:47.565 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-05-08T15:46:47.567 INFO:tasks.ceph.mon.d:Started 2024-05-08T15:46:47.567 INFO:tasks.ceph.mon.e:Restarting daemon 2024-05-08T15:46:47.567 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-05-08T15:46:47.570 INFO:tasks.ceph.mon.e:Started 2024-05-08T15:46:47.570 INFO:tasks.ceph.mon.f:Restarting daemon 2024-05-08T15:46:47.570 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i f 2024-05-08T15:46:47.596 INFO:tasks.ceph.mon.f:Started 2024-05-08T15:46:47.596 INFO:tasks.ceph.mon.g:Restarting daemon 2024-05-08T15:46:47.596 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i g 2024-05-08T15:46:47.598 INFO:tasks.ceph.mon.g:Started 2024-05-08T15:46:47.598 INFO:tasks.ceph.mon.h:Restarting daemon 2024-05-08T15:46:47.598 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i h 2024-05-08T15:46:47.600 INFO:tasks.ceph.mon.h:Started 2024-05-08T15:46:47.600 INFO:tasks.ceph.mon.i:Restarting daemon 2024-05-08T15:46:47.600 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-05-08T15:46:47.602 INFO:tasks.ceph.mon.i:Started 2024-05-08T15:46:47.602 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T15:46:47.603 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T15:46:47.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T15:46:47.606 INFO:tasks.ceph.mgr.x:Started 2024-05-08T15:46:47.606 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T15:46:47.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.756+0000 7fb4ac633640 10 monclient: get_monmap_and_config 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.756+0000 7fb4ac633640 10 monclient: build_initial_monmap 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.756+0000 7fb4ac633640 10 monclient: monmap: 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:47.757660+0000 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:47.757660+0000 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:47.754 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:47.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 1 Processor -- start 2024-05-08T15:46:47.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 1 -- start start 2024-05-08T15:46:47.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 10 monclient: init 2024-05-08T15:46:47.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:47.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 10 monclient: _add_conns ranks=[5,6,7,4,2,1,3,8,0] 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 10 monclient(hunting): picked mon.noname-h con 0x7fb4a40a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 10 monclient(hunting): picked mon.noname-d con 0x7fb4a40a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 10 monclient(hunting): picked mon.noname-i con 0x7fb4a413e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.757+0000 7fb4ac633640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a40a1660 con 0x7fb4a40a53a0 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4ac633640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a40a28f0 con 0x7fb4a40a5780 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4ac633640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a40a2bd0 con 0x7fb4a413e310 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4ac633640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4ac633640 10 monclient(hunting): authenticate will time out at 815.872629s 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4a9bce640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fb4a40a5780 0x7fb4a413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:37124/0 (socket says 172.21.15.7:37124) 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4aa3cf640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fb4a40a53a0 0x7fb4a40a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:38786/0 (socket says 172.21.15.7:38786) 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4a9bce640 1 -- 172.21.15.7:0/1265101240 learned_addr learned my addr 172.21.15.7:0/1265101240 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:46:47.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:47.758+0000 7fb4aabd0640 1 --1- 172.21.15.7:0/1265101240 >> v1:172.21.15.81:6792/0 conn(0x7fb4a413e310 0x7fb4a4140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:46608/0 (socket says 172.21.15.7:46608) 2024-05-08T15:46:50.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient(hunting): tick 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 1 monclient(hunting): continuing hunt 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 1 -- 172.21.15.7:0/1265101240 >> v1:172.21.15.81:6792/0 conn(0x7fb4a413e310 legacy=0x7fb4a4140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 1 -- 172.21.15.7:0/1265101240 >> v1:172.21.15.7:6792/0 conn(0x7fb4a40a5780 legacy=0x7fb4a413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 1 -- 172.21.15.7:0/1265101240 >> v1:172.21.15.81:6791/0 conn(0x7fb4a40a53a0 legacy=0x7fb4a40a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient: _add_conns ranks=[3,0,1,8,4,2] 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient(hunting): picked mon.noname-g con 0x7fb4880037e0 addr v1:172.21.15.81:6790/0 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient(hunting): picked mon.noname-a con 0x7fb488007470 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient(hunting): picked mon.noname-f con 0x7fb48800b200 addr v1:172.21.15.81:6789/0 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 1 -- 172.21.15.7:0/1265101240 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a40a6730 con 0x7fb488007470 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 1 -- 172.21.15.7:0/1265101240 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb488010010 con 0x7fb48800b200 2024-05-08T15:46:50.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 1 -- 172.21.15.7:0/1265101240 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb488011280 con 0x7fb4880037e0 2024-05-08T15:46:50.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:50.758+0000 7fb49bfff640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405206873 0 0) 0x7fb4a40a6730 con 0x7fb488007470 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient(hunting): my global_id is 4104 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb490003520 con 0x7fb488007470 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 303467260 0 0) 0x7fb490003520 con 0x7fb488007470 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 >> v1:172.21.15.81:6790/0 conn(0x7fb4880037e0 legacy=0x7fb488005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 >> v1:172.21.15.81:6789/0 conn(0x7fb48800b200 legacy=0x7fb48800d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: _finish_hunting 0 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 monclient: found mon.noname-a 2024-05-08T15:46:52.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4aabd0640 1 -- 172.21.15.7:0/1265101240 reap_dead start 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4880124f0 con 0x7fb488007470 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: _finish_auth 0 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: _check_auth_tickets 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4940017f0 con 0x7fb488007470 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: dump: 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.926 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4ac633640 5 monclient: authenticate success, global_id 4104 2024-05-08T15:46:52.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4ac633640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb494001b10 con 0x7fb488007470 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 1 -- 172.21.15.7:0/1265101240 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb494001d40 con 0x7fb488007470 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4a93cd640 10 monclient: dump: 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:52.928 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4ac633640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.927+0000 7fb4ac633640 4 set_mon_vals no callback set 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.928+0000 7fb4ac633640 10 monclient: shutdown 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.928+0000 7fb4ac633640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.928+0000 7fb4ac633640 1 -- 172.21.15.7:0/1265101240 >> v1:172.21.15.7:6789/0 conn(0x7fb488007470 legacy=0x7fb488009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.928+0000 7fb4ac633640 1 -- 172.21.15.7:0/1265101240 shutdown_connections 2024-05-08T15:46:52.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.928+0000 7fb4ac633640 1 -- 172.21.15.7:0/1265101240 >> 172.21.15.7:0/1265101240 conn(0x7fb4a40a9fa0 msgr2=0x7fb4a40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.928+0000 7fb4ac633640 1 -- 172.21.15.7:0/1265101240 shutdown_connections 2024-05-08T15:46:52.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.928+0000 7fb4ac633640 1 -- 172.21.15.7:0/1265101240 wait complete. 2024-05-08T15:46:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.929+0000 7fb4ac633640 10 monclient: build_initial_monmap 2024-05-08T15:46:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.929+0000 7fb4ac633640 10 monclient: monmap: 2024-05-08T15:46:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:46:52.932 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.929+0000 7fb4ac633640 1 Processor -- start 2024-05-08T15:46:52.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 1 -- start start 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient: init 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient: _add_conns ranks=[0,1,8,4,2,6,3,5,7] 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient(hunting): picked mon.noname-a con 0x7fb488007470 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient(hunting): picked mon.noname-f con 0x7fb4a4121960 addr v1:172.21.15.81:6789/0 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient(hunting): picked mon.noname-e con 0x7fb4a41cc670 addr v1:172.21.15.7:6793/0 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a41230f0 con 0x7fb488007470 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a409f3c0 con 0x7fb4a4121960 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4a4122850 con 0x7fb4a41cc670 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4ac633640 10 monclient(hunting): authenticate will time out at 821.045208s 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4aa3cf640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb488007470 0x7fb4a4137750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35076/0 (socket says 172.21.15.7:35076) 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.930+0000 7fb4aa3cf640 1 -- 172.21.15.7:0/2625875034 learned_addr learned my addr 172.21.15.7:0/2625875034 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:46:52.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2915897027 0 0) 0x7fb4a409f3c0 con 0x7fb4a4121960 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): my global_id is 4105 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4880101e0 con 0x7fb4a4121960 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507495392 0 0) 0x7fb4a4122850 con 0x7fb4a41cc670 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): my global_id is 4121 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a409f3c0 con 0x7fb4a41cc670 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3797670518 0 0) 0x7fb4a41230f0 con 0x7fb488007470 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:52.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): my global_id is 4113 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a4122850 con 0x7fb488007470 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2378852391 0 0) 0x7fb4880101e0 con 0x7fb4a4121960 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4a41230f0 con 0x7fb4a4121960 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb494001cc0 con 0x7fb4a4121960 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): dump: 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.936 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1946175010 0 0) 0x7fb4a409f3c0 con 0x7fb4a41cc670 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4880101e0 con 0x7fb4a41cc670 2024-05-08T15:46:52.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb49c001b00 con 0x7fb4a41cc670 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): dump: 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:52.938 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 137460817 0 0) 0x7fb4a4122850 con 0x7fb488007470 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4a409f3c0 con 0x7fb488007470 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4a00019c0 con 0x7fb488007470 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.935+0000 7fb49affd640 10 monclient(hunting): dump: 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.939 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:52.940 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3374070400 0 0) 0x7fb4a409f3c0 con 0x7fb488007470 2024-05-08T15:46:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 >> v1:172.21.15.7:6793/0 conn(0x7fb4a41cc670 legacy=0x7fb4a41d50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 >> v1:172.21.15.81:6789/0 conn(0x7fb4a4121960 legacy=0x7fb4a41d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 10 monclient: _finish_hunting 0 2024-05-08T15:46:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 1 monclient: found mon.a 2024-05-08T15:46:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:52.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4a4137e60 con 0x7fb488007470 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 10 monclient: _finish_auth 0 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 10 monclient: _check_auth_tickets 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb4ac633640 5 monclient: authenticate success, global_id 4113 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb4ac633640 10 monclient: _renew_subs 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb4ac633640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb4a41d67e0 con 0x7fb488007470 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb4ac633640 10 monclient: _renew_subs 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.938+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb4a0001ca0 con 0x7fb488007470 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb4ac633640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fb4a41d6b90 con 0x7fb488007470 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb49affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4a0004dd0 con 0x7fb488007470 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb49affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb49affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb4a8bcc640 4 set_mon_vals no callback set 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb49affd640 10 monclient: dump: 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fb4a00053c0 con 0x7fb488007470 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.939+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (unknown 3892747063 0 0) 0x7fb4a0006af0 con 0x7fb488007470 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.940+0000 7fb4ac633640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.940+0000 7fb4ac633640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.940+0000 7fb4ac633640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.940+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4a406dd30 con 0x7fb488007470 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.942+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fb4a406dd30 con 0x7fb488007470 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.942+0000 7fb49affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:52.942+0000 7fb49affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:53.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.013+0000 7fb4ac633640 10 start_mon_command cmd=[{"prefix": "osd crush tunables", "profile": "default"}] 2024-05-08T15:46:53.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.013+0000 7fb4ac633640 10 monclient: _send_command 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-05-08T15:46:53.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.013+0000 7fb4ac633640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:46:53.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.013+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb4a41ccaa0 con 0x7fb488007470 2024-05-08T15:46:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.016+0000 7fb49affd640 1 -- 172.21.15.7:0/2625875034 <== mon.0 v1:172.21.15.7:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (unknown 2277712955 0 0) 0x7fb4a00059e0 con 0x7fb488007470 2024-05-08T15:46:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.016+0000 7fb49affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd crush tunables", "profile": "default"}] 2024-05-08T15:46:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.016+0000 7fb49affd640 10 monclient: _finish_command 2 = system:0 adjusted tunables profile to default 2024-05-08T15:46:53.012 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-05-08T15:46:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.016+0000 7fb4ac633640 10 monclient: shutdown 2024-05-08T15:46:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.016+0000 7fb4ac633640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.016+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 >> v1:172.21.15.7:6789/0 conn(0x7fb488007470 legacy=0x7fb4a4137750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.017+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 shutdown_connections 2024-05-08T15:46:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.017+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 >> 172.21.15.7:0/2625875034 conn(0x7fb4a40a9fa0 msgr2=0x7fb4a413abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.017+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 shutdown_connections 2024-05-08T15:46:53.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:53.017+0000 7fb4ac633640 1 -- 172.21.15.7:0/2625875034 wait complete. 2024-05-08T15:46:53.024 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T15:46:53.024 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T15:46:53.024 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:53.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T15:46:53.046 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:53.050+0000 7fcfa3c21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T15:46:53.051 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:53.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T15:46:53.117 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-05-08T15:46:53.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T15:46:53.183 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:53.183 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T15:46:53.202 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:53.206+0000 7fcfa3c21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T15:46:53.209 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:53.209 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T15:46:53.258 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:53.261+0000 7fcfa3c21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T15:46:53.274 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-05-08T15:46:53.274 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T15:46:53.339 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 8f2a2966-1786-4529-bf11-c65c304769c6 0 2024-05-08T15:46:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.480+0000 7f7fe8b3b640 10 monclient: get_monmap_and_config 2024-05-08T15:46:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.480+0000 7f7fe8b3b640 10 monclient: build_initial_monmap 2024-05-08T15:46:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.480+0000 7f7fe8b3b640 10 monclient: monmap: 2024-05-08T15:46:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:53.481524+0000 2024-05-08T15:46:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:53.481524+0000 2024-05-08T15:46:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:53.478 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:53.479 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:53.479 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 1 Processor -- start 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 1 -- start start 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient: init 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient: _add_conns ranks=[8,4,7,1,2,0,5,3,6] 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient(hunting): picked mon.noname-e con 0x7f7fe40995f0 addr v1:172.21.15.7:6793/0 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient(hunting): picked mon.noname-c con 0x7f7fe4135ea0 addr v1:172.21.15.7:6791/0 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient(hunting): picked mon.noname-i con 0x7f7fe41362f0 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe409a460 con 0x7f7fe4135ea0 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe409a720 con 0x7f7fe41362f0 2024-05-08T15:46:53.490 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe409aa00 con 0x7f7fe40995f0 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe8b3b640 10 monclient(hunting): authenticate will time out at 804.702087s 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe2d9d640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7f7fe41362f0 0x7f7fe413e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.81:54394/0 (socket says 172.21.15.81:54394) 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.481+0000 7f7fe2d9d640 1 -- 172.21.15.81:0/3872692893 learned_addr learned my addr 172.21.15.81:0/3872692893 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4134692892 0 0) 0x7f7fe409a720 con 0x7f7fe41362f0 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient(hunting): my global_id is 4111 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fcc003520 con 0x7f7fe41362f0 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2310446413 0 0) 0x7f7fcc003520 con 0x7f7fe41362f0 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 >> v1:172.21.15.7:6793/0 conn(0x7f7fe40995f0 legacy=0x7f7fe40999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 >> v1:172.21.15.7:6791/0 conn(0x7f7fe4135ea0 legacy=0x7f7fe413a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: _finish_hunting 0 2024-05-08T15:46:53.491 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 monclient: found mon.noname-i 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.81:6792/0 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe4067460 con 0x7f7fe41362f0 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: _finish_auth 0 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: _check_auth_tickets 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7fdc001b30 con 0x7f7fe41362f0 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe8b3b640 5 monclient: authenticate success, global_id 4111 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: dump: 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.492 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.493 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe8b3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 <== mon.7 v1:172.21.15.81:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7fdc001e90 con 0x7f7fe41362f0 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 1 -- 172.21.15.81:0/3872692893 <== mon.7 v1:172.21.15.81:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7fdc0020c0 con 0x7f7fe41362f0 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe8b3b640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe8b3b640 4 set_mon_vals no callback set 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.482+0000 7f7fe159a640 10 monclient: dump: 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.494 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.495 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 10 monclient: shutdown 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/3872692893 >> v1:172.21.15.81:6792/0 conn(0x7f7fe41362f0 legacy=0x7f7fe413e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/3872692893 shutdown_connections 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/3872692893 >> 172.21.15.81:0/3872692893 conn(0x7f7fe40a2de0 msgr2=0x7f7fe40a31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/3872692893 shutdown_connections 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/3872692893 wait complete. 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 10 monclient: build_initial_monmap 2024-05-08T15:46:53.496 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.483+0000 7f7fe8b3b640 10 monclient: monmap: 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 0.000000 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:created 0.000000 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 1 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:53.497 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 1 Processor -- start 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 1 -- start start 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient: init 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient: _add_conns ranks=[5,7,3,8,1,6,0,2,4] 2024-05-08T15:46:53.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient(hunting): picked mon.noname-h con 0x7f7fe40995f0 addr v1:172.21.15.81:6791/0 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient(hunting): picked mon.noname-i con 0x7f7fe4135ea0 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient(hunting): picked mon.noname-g con 0x7f7fe41362f0 addr v1:172.21.15.81:6790/0 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe409ebd0 con 0x7f7fe41362f0 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe409f340 con 0x7f7fe40995f0 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fe40a0090 con 0x7f7fe4135ea0 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe8b3b640 10 monclient(hunting): authenticate will time out at 804.704995s 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe1d9b640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7f7fe4135ea0 0x7f7fe41d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.81:54406/0 (socket says 172.21.15.81:54406) 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe259c640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7f7fe40995f0 0x7f7fe413ecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.81:60060/0 (socket says 172.21.15.81:60060) 2024-05-08T15:46:53.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.484+0000 7f7fe1d9b640 1 -- 172.21.15.81:0/1616514090 learned_addr learned my addr 172.21.15.81:0/1616514090 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 880354419 0 0) 0x7f7fe40a0090 con 0x7f7fe4135ea0 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): my global_id is 4120 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fc4003520 con 0x7f7fe4135ea0 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060787413 0 0) 0x7f7fe409f340 con 0x7f7fe40995f0 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): my global_id is 4109 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fe40a0090 con 0x7f7fe40995f0 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1464521332 0 0) 0x7f7fc4003520 con 0x7f7fe4135ea0 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7fe409f340 con 0x7f7fe4135ea0 2024-05-08T15:46:53.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7fd8001b10 con 0x7f7fe4135ea0 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): dump: 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.501 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3726117932 0 0) 0x7f7fe40a0090 con 0x7f7fe40995f0 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7fc4003520 con 0x7f7fe40995f0 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7fd40018d0 con 0x7f7fe40995f0 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient(hunting): dump: 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.502 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2113983045 0 0) 0x7f7fe409f340 con 0x7f7fe4135ea0 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 >> v1:172.21.15.81:6791/0 conn(0x7f7fe40995f0 legacy=0x7f7fe413ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 >> v1:172.21.15.81:6790/0 conn(0x7f7fe41362f0 legacy=0x7f7fe41d53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 monclient: found mon.i 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe4131fc0 con 0x7f7fe4135ea0 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe8b3b640 5 monclient: authenticate success, global_id 4120 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe8b3b640 10 monclient: _renew_subs 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe8b3b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe409fc00 con 0x7f7fe4135ea0 2024-05-08T15:46:53.504 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe8b3b640 10 monclient: _renew_subs 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe8b3b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f7fe413f5a0 con 0x7f7fe4135ea0 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7fd80038c0 con 0x7f7fe4135ea0 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7fd8003e30 con 0x7f7fe4135ea0 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fe0d99640 4 set_mon_vals no callback set 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 10 monclient: dump: 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.505 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f7fd80057c0 con 0x7f7fe4135ea0 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.485+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (unknown 2309151905 0 0) 0x7f7fd8006c50 con 0x7f7fe4135ea0 2024-05-08T15:46:53.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.486+0000 7f7fe8b3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.486+0000 7f7fe8b3b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.486+0000 7f7fe8b3b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:53.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.486+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc8005080 con 0x7f7fe4135ea0 2024-05-08T15:46:53.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.488+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f7fd8005ab0 con 0x7f7fe4135ea0 2024-05-08T15:46:53.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.488+0000 7f7fd2ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.488+0000 7f7fd2ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:53.522 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:53.525+0000 7fcfa3c21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T15:46:53.555 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.559+0000 7f7fe8b3b640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "8f2a2966-1786-4529-bf11-c65c304769c6", "id": 0}] 2024-05-08T15:46:53.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.559+0000 7f7fe8b3b640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "8f2a2966-1786-4529-bf11-c65c304769c6", "id": 0}] 2024-05-08T15:46:53.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.559+0000 7f7fe8b3b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:53.556 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.559+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "osd new", "uuid": "8f2a2966-1786-4529-bf11-c65c304769c6", "id": 0} v 0) -- 0x7f7fc8005370 con 0x7f7fe4135ea0 2024-05-08T15:46:53.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.565+0000 7f7fd2ffd640 1 -- 172.21.15.81:0/1616514090 <== mon.7 v1:172.21.15.81:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8f2a2966-1786-4529-bf11-c65c304769c6", "id": 0}]=0 v3) ==== 112+0+2 (unknown 1978243476 0 732707515) 0x7f7fd8005de0 con 0x7f7fe4135ea0 2024-05-08T15:46:53.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.565+0000 7f7fd2ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "8f2a2966-1786-4529-bf11-c65c304769c6", "id": 0}] 2024-05-08T15:46:53.561 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.565+0000 7f7fd2ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:53.562 INFO:teuthology.orchestra.run.smithi081.stdout:0 2024-05-08T15:46:53.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.566+0000 7f7fe8b3b640 10 monclient: shutdown 2024-05-08T15:46:53.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.566+0000 7f7fe8b3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:53.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.566+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 >> v1:172.21.15.81:6792/0 conn(0x7f7fe4135ea0 legacy=0x7f7fe41d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.566+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 shutdown_connections 2024-05-08T15:46:53.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.566+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 >> 172.21.15.81:0/1616514090 conn(0x7f7fe40a2de0 msgr2=0x7f7fe40a11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:53.563 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.567+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 shutdown_connections 2024-05-08T15:46:53.564 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.567+0000 7f7fe8b3b640 1 -- 172.21.15.81:0/1616514090 wait complete. 2024-05-08T15:46:53.571 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:53.575+0000 7fcfa3c21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T15:46:53.577 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 68812ea6-7e31-48c1-a823-1748f6eda800 1 2024-05-08T15:46:53.674 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.677+0000 7f4e5cf07640 10 monclient: get_monmap_and_config 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.677+0000 7f4e5cf07640 10 monclient: build_initial_monmap 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.678+0000 7f4e5cf07640 10 monclient: monmap: 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:53.678984+0000 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:53.678984+0000 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:53.675 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.678+0000 7f4e5cf07640 1 Processor -- start 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.678+0000 7f4e5cf07640 1 -- start start 2024-05-08T15:46:53.676 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.678+0000 7f4e5cf07640 10 monclient: init 2024-05-08T15:46:53.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:53.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 10 monclient: _add_conns ranks=[1,4,3,2,0,7,6,5,8] 2024-05-08T15:46:53.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 10 monclient(hunting): picked mon.noname-f con 0x7f4e580a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:46:53.684 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 10 monclient(hunting): picked mon.noname-c con 0x7f4e580a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 10 monclient(hunting): picked mon.noname-g con 0x7f4e5813e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e580a1660 con 0x7f4e580a53a0 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e580a28f0 con 0x7f4e5813e310 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e580a2bd0 con 0x7f4e580a5780 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 10 monclient(hunting): authenticate will time out at 804.899588s 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e56d9d640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f4e5813e310 0x7f4e58140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.81:33578/0 (socket says 172.21.15.81:33578) 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e56d9d640 1 -- 172.21.15.81:0/4236911168 learned_addr learned my addr 172.21.15.81:0/4236911168 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46215308 0 0) 0x7f4e580a1660 con 0x7f4e580a53a0 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.685 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient(hunting): my global_id is 4114 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e44003520 con 0x7f4e580a53a0 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4027369155 0 0) 0x7f4e44003520 con 0x7f4e580a53a0 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 >> v1:172.21.15.7:6791/0 conn(0x7f4e580a5780 legacy=0x7f4e5813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 >> v1:172.21.15.81:6790/0 conn(0x7f4e5813e310 legacy=0x7f4e58140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient: _finish_hunting 0 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 monclient: found mon.noname-f 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e580a6730 con 0x7f4e580a53a0 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient: _finish_auth 0 2024-05-08T15:46:53.686 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient: _check_auth_tickets 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4e480017b0 con 0x7f4e580a53a0 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 5 monclient: authenticate success, global_id 4114 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5559a640 10 monclient: dump: 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.687 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.688 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.688 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.688 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.688 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.688 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.688 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.688 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.679+0000 7f4e5cf07640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4e48001b10 con 0x7f4e580a53a0 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5559a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5559a640 1 -- 172.21.15.81:0/4236911168 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4e48001d40 con 0x7f4e580a53a0 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5559a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5559a640 10 monclient: dump: 2024-05-08T15:46:53.689 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.690 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 4 set_mon_vals no callback set 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 10 monclient: shutdown 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 1 -- 172.21.15.81:0/4236911168 >> v1:172.21.15.81:6789/0 conn(0x7f4e580a53a0 legacy=0x7f4e580a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 1 -- 172.21.15.81:0/4236911168 shutdown_connections 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 1 -- 172.21.15.81:0/4236911168 >> 172.21.15.81:0/4236911168 conn(0x7f4e580a9fa0 msgr2=0x7f4e580aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 1 -- 172.21.15.81:0/4236911168 shutdown_connections 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.680+0000 7f4e5cf07640 1 -- 172.21.15.81:0/4236911168 wait complete. 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.681+0000 7f4e5cf07640 10 monclient: build_initial_monmap 2024-05-08T15:46:53.691 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.681+0000 7f4e5cf07640 10 monclient: monmap: 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 0.000000 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:created 0.000000 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 1 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:53.692 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.681+0000 7f4e5cf07640 1 Processor -- start 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.681+0000 7f4e5cf07640 1 -- start start 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.681+0000 7f4e5cf07640 10 monclient: init 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 10 monclient: _add_conns ranks=[5,8,0,6,2,4,1,3,7] 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 10 monclient(hunting): picked mon.noname-h con 0x7f4e580a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 10 monclient(hunting): picked mon.noname-e con 0x7f4e580a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 10 monclient(hunting): picked mon.noname-a con 0x7f4e5813e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58133570 con 0x7f4e5813e310 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58134190 con 0x7f4e580a53a0 2024-05-08T15:46:53.693 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e58134470 con 0x7f4e580a5780 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5cf07640 10 monclient(hunting): authenticate will time out at 804.902594s 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5659c640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7f4e580a53a0 0x7f4e58137f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.81:60074/0 (socket says 172.21.15.81:60074) 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e5659c640 1 -- 172.21.15.81:0/351182251 learned_addr learned my addr 172.21.15.81:0/351182251 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1646194451 0 0) 0x7f4e58134190 con 0x7f4e580a53a0 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): my global_id is 4118 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e38003520 con 0x7f4e580a53a0 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2058533127 0 0) 0x7f4e58133570 con 0x7f4e5813e310 2024-05-08T15:46:53.694 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): my global_id is 4122 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e58134190 con 0x7f4e5813e310 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3141450313 0 0) 0x7f4e58134470 con 0x7f4e580a5780 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): my global_id is 4139 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e58133570 con 0x7f4e580a5780 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3297576781 0 0) 0x7f4e38003520 con 0x7f4e580a53a0 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e58134470 con 0x7f4e580a53a0 2024-05-08T15:46:53.695 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4e48001770 con 0x7f4e580a53a0 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): dump: 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.696 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.682+0000 7f4e4effd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 742809242 0 0) 0x7f4e58134190 con 0x7f4e5813e310 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e38003520 con 0x7f4e5813e310 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 761821487 0 0) 0x7f4e58133570 con 0x7f4e580a5780 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e58134190 con 0x7f4e580a5780 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4e50001ba0 con 0x7f4e5813e310 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:53.697 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient(hunting): dump: 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.698 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3494227315 0 0) 0x7f4e58134470 con 0x7f4e580a53a0 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 >> v1:172.21.15.7:6793/0 conn(0x7f4e580a5780 legacy=0x7f4e581d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 >> v1:172.21.15.7:6789/0 conn(0x7f4e5813e310 legacy=0x7f4e581d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: _finish_hunting 0 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 monclient: found mon.h 2024-05-08T15:46:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e58134750 con 0x7f4e580a53a0 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: _finish_auth 0 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: _check_auth_tickets 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e5cf07640 5 monclient: authenticate success, global_id 4118 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e5cf07640 10 monclient: _renew_subs 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e5cf07640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.81:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4e58138670 con 0x7f4e580a53a0 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e5cf07640 10 monclient: _renew_subs 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e5cf07640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.81:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f4e581ccd10 con 0x7f4e580a53a0 2024-05-08T15:46:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4e48003d70 con 0x7f4e580a53a0 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4e48004260 con 0x7f4e580a53a0 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 10 monclient: dump: 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e54d99640 4 set_mon_vals no callback set 2024-05-08T15:46:53.702 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f4e48005c50 con 0x7f4e580a53a0 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.683+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 8 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (unknown 3322973778 0 0) 0x7f4e48006ce0 con 0x7f4e580a53a0 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.684+0000 7f4e5cf07640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.684+0000 7f4e5cf07640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.684+0000 7f4e5cf07640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.684+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c005080 con 0x7f4e580a53a0 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.686+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f4e48004810 con 0x7f4e580a53a0 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.686+0000 7f4e4effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.686+0000 7f4e4effd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:53.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.773+0000 7f4e5cf07640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "68812ea6-7e31-48c1-a823-1748f6eda800", "id": 1}] 2024-05-08T15:46:53.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.773+0000 7f4e5cf07640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "68812ea6-7e31-48c1-a823-1748f6eda800", "id": 1}] 2024-05-08T15:46:53.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.773+0000 7f4e5cf07640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:53.770 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.773+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "osd new", "uuid": "68812ea6-7e31-48c1-a823-1748f6eda800", "id": 1} v 0) -- 0x7f4e3c005370 con 0x7f4e580a53a0 2024-05-08T15:46:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.781+0000 7f4e4effd640 1 -- 172.21.15.81:0/351182251 <== mon.5 v1:172.21.15.81:6791/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "68812ea6-7e31-48c1-a823-1748f6eda800", "id": 1}]=0 v4) ==== 112+0+2 (unknown 2907330743 0 940483276) 0x7f4e48005f40 con 0x7f4e580a53a0 2024-05-08T15:46:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.781+0000 7f4e4effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "68812ea6-7e31-48c1-a823-1748f6eda800", "id": 1}] 2024-05-08T15:46:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.781+0000 7f4e4effd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:53.778 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-05-08T15:46:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.782+0000 7f4e5cf07640 10 monclient: shutdown 2024-05-08T15:46:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.782+0000 7f4e5cf07640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.782+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 >> v1:172.21.15.81:6791/0 conn(0x7f4e580a53a0 legacy=0x7f4e58137f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.782+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 shutdown_connections 2024-05-08T15:46:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.782+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 >> 172.21.15.81:0/351182251 conn(0x7f4e580a9fa0 msgr2=0x7f4e580a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.782+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 shutdown_connections 2024-05-08T15:46:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.782+0000 7f4e5cf07640 1 -- 172.21.15.81:0/351182251 wait complete. 2024-05-08T15:46:53.789 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 3ca973c1-ee0b-43e5-a644-a8e110676b39 2 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.889+0000 7fa78f3d4640 10 monclient: get_monmap_and_config 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.889+0000 7fa78f3d4640 10 monclient: build_initial_monmap 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.889+0000 7fa78f3d4640 10 monclient: monmap: 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:53.890622+0000 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:53.890622+0000 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.886 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:53.887 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 1 Processor -- start 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 1 -- start start 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 10 monclient: init 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 10 monclient: _add_conns ranks=[2,0,7,5,6,3,4,1,8] 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 10 monclient(hunting): picked mon.noname-b con 0x7fa7880a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 10 monclient(hunting): picked mon.noname-a con 0x7fa7880a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 10 monclient(hunting): picked mon.noname-i con 0x7fa78813e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7880a1660 con 0x7fa7880a5780 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.890+0000 7fa78f3d4640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7880a28f0 con 0x7fa7880a53a0 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78f3d4640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7880a2bd0 con 0x7fa78813e310 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78f3d4640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:53.895 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78f3d4640 10 monclient(hunting): authenticate will time out at 805.111278s 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78d971640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7fa78813e310 0x7fa788140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.81:54420/0 (socket says 172.21.15.81:54420) 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78d971640 1 -- 172.21.15.81:0/2834780687 learned_addr learned my addr 172.21.15.81:0/2834780687 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2898457334 0 0) 0x7fa7880a2bd0 con 0x7fa78813e310 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): my global_id is 4129 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa774003520 con 0x7fa78813e310 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415830852 0 0) 0x7fa7880a1660 con 0x7fa7880a5780 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient(hunting): my global_id is 4131 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7880a2bd0 con 0x7fa7880a5780 2024-05-08T15:46:53.896 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2711471352 0 0) 0x7fa774003520 con 0x7fa78813e310 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 >> v1:172.21.15.7:6790/0 conn(0x7fa7880a53a0 legacy=0x7fa7880a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 >> v1:172.21.15.7:6789/0 conn(0x7fa7880a5780 legacy=0x7fa78813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 monclient: found mon.noname-i 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.81:6792/0 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7880a6730 con 0x7fa78813e310 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: _finish_auth 0 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78f3d4640 5 monclient: authenticate success, global_id 4129 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa780001b30 con 0x7fa78813e310 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.897 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: dump: 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.898 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78f3d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 <== mon.7 v1:172.21.15.81:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa780001d90 con 0x7fa78813e310 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78f3d4640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa78f3d4640 4 set_mon_vals no callback set 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 1 -- 172.21.15.81:0/2834780687 <== mon.7 v1:172.21.15.81:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa780004f60 con 0x7fa78813e310 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.891+0000 7fa77ffff640 10 monclient: dump: 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.899 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.892+0000 7fa78f3d4640 10 monclient: shutdown 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.892+0000 7fa78f3d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:53.900 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.892+0000 7fa78f3d4640 1 -- 172.21.15.81:0/2834780687 >> v1:172.21.15.81:6792/0 conn(0x7fa78813e310 legacy=0x7fa788140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.892+0000 7fa78f3d4640 1 -- 172.21.15.81:0/2834780687 shutdown_connections 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.892+0000 7fa78f3d4640 1 -- 172.21.15.81:0/2834780687 >> 172.21.15.81:0/2834780687 conn(0x7fa7880a9fa0 msgr2=0x7fa7880aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.892+0000 7fa78f3d4640 1 -- 172.21.15.81:0/2834780687 shutdown_connections 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.892+0000 7fa78f3d4640 1 -- 172.21.15.81:0/2834780687 wait complete. 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient: build_initial_monmap 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient: monmap: 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 0.000000 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:created 0.000000 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 1 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:53.901 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 1 Processor -- start 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 1 -- start start 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient: init 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient: _add_conns ranks=[4,7,1,6,8,5,0,3,2] 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient(hunting): picked mon.noname-c con 0x7fa7880a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:46:53.902 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient(hunting): picked mon.noname-i con 0x7fa7880a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient(hunting): picked mon.noname-f con 0x7fa78813e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7880a43c0 con 0x7fa78813e310 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa788132f00 con 0x7fa7880a53a0 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa788134890 con 0x7fa7880a5780 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.893+0000 7fa78f3d4640 10 monclient(hunting): authenticate will time out at 805.114209s 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa78c96f640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7fa7880a5780 0x7fa7881d1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.81:54430/0 (socket says 172.21.15.81:54430) 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa78c96f640 1 -- 172.21.15.81:0/88549559 learned_addr learned my addr 172.21.15.81:0/88549559 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1135867626 0 0) 0x7fa7880a43c0 con 0x7fa78813e310 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): my global_id is 4123 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa76c003520 con 0x7fa78813e310 2024-05-08T15:46:53.903 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2711125206 0 0) 0x7fa788134890 con 0x7fa7880a5780 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): my global_id is 4138 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7880a43c0 con 0x7fa7880a5780 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1783948839 0 0) 0x7fa788132f00 con 0x7fa7880a53a0 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): my global_id is 4126 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa788134890 con 0x7fa7880a53a0 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3992686795 0 0) 0x7fa76c003520 con 0x7fa78813e310 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa788132f00 con 0x7fa78813e310 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1320043643 0 0) 0x7fa7880a43c0 con 0x7fa7880a5780 2024-05-08T15:46:53.904 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa76c003520 con 0x7fa7880a5780 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa780001f40 con 0x7fa78813e310 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): dump: 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.905 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa778001a90 con 0x7fa7880a5780 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient(hunting): dump: 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.906 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1718767605 0 0) 0x7fa788132f00 con 0x7fa78813e310 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 >> v1:172.21.15.81:6792/0 conn(0x7fa7880a5780 legacy=0x7fa7881d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 >> v1:172.21.15.7:6791/0 conn(0x7fa7880a53a0 legacy=0x7fa78812e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.907 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: _finish_hunting 0 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 monclient: found mon.f 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa788134b70 con 0x7fa78813e310 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: _finish_auth 0 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: _check_auth_tickets 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa78f3d4640 5 monclient: authenticate success, global_id 4123 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa78f3d4640 10 monclient: _renew_subs 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa78f3d4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa7881339a0 con 0x7fa78813e310 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa780001b10 con 0x7fa78813e310 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:53.908 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa780005490 con 0x7fa78813e310 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 10 monclient: dump: 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:53.909 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fa7800059c0 con 0x7fa78813e310 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.895+0000 7fa78f3d4640 10 monclient: _renew_subs 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.894+0000 7fa77f7fe640 4 set_mon_vals no callback set 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.895+0000 7fa78f3d4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.895+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa7881386d0 con 0x7fa78813e310 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.895+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (unknown 2279007432 0 0) 0x7fa7800062b0 con 0x7fa78813e310 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.896+0000 7fa78f3d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.896+0000 7fa78f3d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.896+0000 7fa78f3d4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.896+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7880a8f80 con 0x7fa78813e310 2024-05-08T15:46:53.910 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.898+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fa7880a8f80 con 0x7fa78813e310 2024-05-08T15:46:53.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.898+0000 7fa77dffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:53.911 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.898+0000 7fa77dffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:53.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.979+0000 7fa78f3d4640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "3ca973c1-ee0b-43e5-a644-a8e110676b39", "id": 2}] 2024-05-08T15:46:53.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.979+0000 7fa78f3d4640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "3ca973c1-ee0b-43e5-a644-a8e110676b39", "id": 2}] 2024-05-08T15:46:53.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.979+0000 7fa78f3d4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:46:53.976 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.979+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "osd new", "uuid": "3ca973c1-ee0b-43e5-a644-a8e110676b39", "id": 2} v 0) -- 0x7fa788075820 con 0x7fa78813e310 2024-05-08T15:46:53.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.986+0000 7fa77dffb640 1 -- 172.21.15.81:0/88549559 <== mon.1 v1:172.21.15.81:6789/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ca973c1-ee0b-43e5-a644-a8e110676b39", "id": 2}]=0 v5) ==== 112+0+2 (unknown 3422266513 0 216599125) 0x7fa780005e90 con 0x7fa78813e310 2024-05-08T15:46:53.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.986+0000 7fa77dffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "3ca973c1-ee0b-43e5-a644-a8e110676b39", "id": 2}] 2024-05-08T15:46:53.982 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.986+0000 7fa77dffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:53.983 INFO:teuthology.orchestra.run.smithi081.stdout:2 2024-05-08T15:46:53.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.987+0000 7fa78f3d4640 10 monclient: shutdown 2024-05-08T15:46:53.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.987+0000 7fa78f3d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:53.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.987+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 >> v1:172.21.15.81:6789/0 conn(0x7fa78813e310 legacy=0x7fa7881d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:53.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.987+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 shutdown_connections 2024-05-08T15:46:53.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.987+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 >> 172.21.15.81:0/88549559 conn(0x7fa7880a9fa0 msgr2=0x7fa78813eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:53.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.987+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 shutdown_connections 2024-05-08T15:46:53.984 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:53.987+0000 7fa78f3d4640 1 -- 172.21.15.81:0/88549559 wait complete. 2024-05-08T15:46:53.994 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 0420b958-6a66-413a-bb95-fcc848bf9708 3 2024-05-08T15:46:54.091 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.094+0000 7f0f09c3f640 10 monclient: get_monmap_and_config 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.094+0000 7f0f09c3f640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.094+0000 7f0f09c3f640 10 monclient: monmap: 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:54.095869+0000 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:54.095869+0000 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.092 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.093 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.093 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.093 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.093 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.093 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.093 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.093 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.095+0000 7f0f09c3f640 1 Processor -- start 2024-05-08T15:46:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.095+0000 7f0f09c3f640 1 -- start start 2024-05-08T15:46:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.095+0000 7f0f09c3f640 10 monclient: init 2024-05-08T15:46:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 10 monclient: _add_conns ranks=[2,3,0,4,7,1,5,8,6] 2024-05-08T15:46:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 10 monclient(hunting): picked mon.noname-b con 0x7f0f040a1390 addr v1:172.21.15.7:6790/0 2024-05-08T15:46:54.099 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 10 monclient(hunting): picked mon.noname-g con 0x7f0f040a1770 addr v1:172.21.15.81:6790/0 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 10 monclient(hunting): picked mon.noname-a con 0x7f0f040a1b70 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f04136370 con 0x7f0f040a1b70 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f04072650 con 0x7f0f040a1390 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f04071550 con 0x7f0f040a1770 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0f09c3f640 10 monclient(hunting): authenticate will time out at 805.316418s 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0efaffd640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f0f040a1770 0x7f0f0413a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.81:33586/0 (socket says 172.21.15.81:33586) 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.096+0000 7f0efaffd640 1 -- 172.21.15.81:0/2829196735 learned_addr learned my addr 172.21.15.81:0/2829196735 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297720889 0 0) 0x7f0f04136370 con 0x7f0f040a1b70 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient(hunting): my global_id is 4140 2024-05-08T15:46:54.100 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ee4003520 con 0x7f0f040a1b70 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 390835652 0 0) 0x7f0ee4003520 con 0x7f0f040a1b70 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 >> v1:172.21.15.81:6790/0 conn(0x7f0f040a1770 legacy=0x7f0f0413a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 >> v1:172.21.15.7:6790/0 conn(0x7f0f040a1390 legacy=0x7f0f04135c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 monclient: found mon.noname-a 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f04072930 con 0x7f0f040a1b70 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.101 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0efc001b50 con 0x7f0f040a1b70 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f09c3f640 5 monclient: authenticate success, global_id 4140 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f09c3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: dump: 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.102 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.103 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.104 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f09c3f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0efc001db0 con 0x7f0f040a1b70 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f09c3f640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f09c3f640 4 set_mon_vals no callback set 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 1 -- 172.21.15.81:0/2829196735 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0efc003cd0 con 0x7f0f040a1b70 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.097+0000 7f0f02ffd640 10 monclient: dump: 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.105 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.098+0000 7f0f09c3f640 10 monclient: shutdown 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.098+0000 7f0f09c3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.098+0000 7f0f09c3f640 1 -- 172.21.15.81:0/2829196735 >> v1:172.21.15.7:6789/0 conn(0x7f0f040a1b70 legacy=0x7f0f0413dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.106 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.098+0000 7f0f09c3f640 1 -- 172.21.15.81:0/2829196735 shutdown_connections 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.098+0000 7f0f09c3f640 1 -- 172.21.15.81:0/2829196735 >> 172.21.15.81:0/2829196735 conn(0x7f0f040a8360 msgr2=0x7f0f040a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.098+0000 7f0f09c3f640 1 -- 172.21.15.81:0/2829196735 shutdown_connections 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.098+0000 7f0f09c3f640 1 -- 172.21.15.81:0/2829196735 wait complete. 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.099+0000 7f0f09c3f640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.099+0000 7f0f09c3f640 10 monclient: monmap: 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 0.000000 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:created 0.000000 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 1 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.107 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.099+0000 7f0f09c3f640 1 Processor -- start 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.099+0000 7f0f09c3f640 1 -- start start 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.099+0000 7f0f09c3f640 10 monclient: init 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 10 monclient: _add_conns ranks=[8,7,6,1,3,0,2,5,4] 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 10 monclient(hunting): picked mon.noname-e con 0x7f0f040a1390 addr v1:172.21.15.7:6793/0 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 10 monclient(hunting): picked mon.noname-i con 0x7f0f040a1770 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 10 monclient(hunting): picked mon.noname-d con 0x7f0f040a1b70 addr v1:172.21.15.7:6792/0 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f0409d630 con 0x7f0f040a1b70 2024-05-08T15:46:54.108 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f040a20a0 con 0x7f0f040a1770 2024-05-08T15:46:54.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f040717f0 con 0x7f0f040a1390 2024-05-08T15:46:54.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0f09c3f640 10 monclient(hunting): authenticate will time out at 805.320599s 2024-05-08T15:46:54.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0efaffd640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7f0f040a1770 0x7f0f041cf960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.81:54440/0 (socket says 172.21.15.81:54440) 2024-05-08T15:46:54.109 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.100+0000 7f0efaffd640 1 -- 172.21.15.81:0/1984997582 learned_addr learned my addr 172.21.15.81:0/1984997582 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2958034178 0 0) 0x7f0f0409d630 con 0x7f0f040a1b70 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): my global_id is 74103 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ee8003520 con 0x7f0f040a1b70 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2663861952 0 0) 0x7f0f040717f0 con 0x7f0f040a1390 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.112 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): my global_id is 4148 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0f0409d630 con 0x7f0f040a1390 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2088927635 0 0) 0x7f0ee8003520 con 0x7f0f040a1b70 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0f040717f0 con 0x7f0f040a1b70 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 669683762 0 0) 0x7f0f0409d630 con 0x7f0f040a1390 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ee8003520 con 0x7f0f040a1390 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0efc001a50 con 0x7f0f040a1b70 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): dump: 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.113 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ef4001a10 con 0x7f0f040a1390 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.114 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient(hunting): dump: 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.115 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3265876419 0 0) 0x7f0f040717f0 con 0x7f0f040a1b70 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 >> v1:172.21.15.7:6793/0 conn(0x7f0f040a1390 legacy=0x7f0f041ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 >> v1:172.21.15.81:6792/0 conn(0x7f0f040a1770 legacy=0x7f0f041cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 monclient: found mon.d 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f04130c60 con 0x7f0f040a1b70 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f09c3f640 5 monclient: authenticate success, global_id 74103 2024-05-08T15:46:54.116 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f09c3f640 10 monclient: _renew_subs 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f09c3f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0f0409dcf0 con 0x7f0f040a1b70 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f09c3f640 10 monclient: _renew_subs 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f09c3f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f0f0409cd00 con 0x7f0f040a1b70 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0efc001db0 con 0x7f0f040a1b70 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0efc004280 con 0x7f0f040a1b70 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f027fc640 4 set_mon_vals no callback set 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 10 monclient: dump: 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.117 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f0efc0047b0 con 0x7f0f040a1b70 2024-05-08T15:46:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.104+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (unknown 3699076142 0 0) 0x7f0efc005d20 con 0x7f0f040a1b70 2024-05-08T15:46:54.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.105+0000 7f0f09c3f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.105+0000 7f0f09c3f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.105+0000 7f0f09c3f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:54.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.105+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0edc005080 con 0x7f0f040a1b70 2024-05-08T15:46:54.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.108+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f0efc005890 con 0x7f0f040a1b70 2024-05-08T15:46:54.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.108+0000 7f0f00ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.119 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.108+0000 7f0f00ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:54.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.180+0000 7f0f09c3f640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "0420b958-6a66-413a-bb95-fcc848bf9708", "id": 3}] 2024-05-08T15:46:54.177 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.180+0000 7f0f09c3f640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "0420b958-6a66-413a-bb95-fcc848bf9708", "id": 3}] 2024-05-08T15:46:54.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.180+0000 7f0f09c3f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:54.178 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.180+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "osd new", "uuid": "0420b958-6a66-413a-bb95-fcc848bf9708", "id": 3} v 0) -- 0x7f0edc005370 con 0x7f0f040a1b70 2024-05-08T15:46:54.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.185+0000 7f0f00ff9640 1 -- 172.21.15.81:0/1984997582 <== mon.6 v1:172.21.15.7:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0420b958-6a66-413a-bb95-fcc848bf9708", "id": 3}]=0 v6) ==== 112+0+2 (unknown 3015081457 0 525046306) 0x7f0efc004aa0 con 0x7f0f040a1b70 2024-05-08T15:46:54.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.185+0000 7f0f00ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "0420b958-6a66-413a-bb95-fcc848bf9708", "id": 3}] 2024-05-08T15:46:54.181 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.185+0000 7f0f00ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:54.181 INFO:teuthology.orchestra.run.smithi081.stdout:3 2024-05-08T15:46:54.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.186+0000 7f0f09c3f640 10 monclient: shutdown 2024-05-08T15:46:54.182 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.186+0000 7f0f09c3f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.186+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 >> v1:172.21.15.7:6792/0 conn(0x7f0f040a1b70 legacy=0x7f0f041d3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.186+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 shutdown_connections 2024-05-08T15:46:54.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.186+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 >> 172.21.15.81:0/1984997582 conn(0x7f0f040a8360 msgr2=0x7f0f040a5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.186+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 shutdown_connections 2024-05-08T15:46:54.183 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.186+0000 7f0f09c3f640 1 -- 172.21.15.81:0/1984997582 wait complete. 2024-05-08T15:46:54.193 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 64485986-c4f5-4899-8a0c-604433e57b46 4 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.295+0000 7f0aecb58640 10 monclient: get_monmap_and_config 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.295+0000 7f0aecb58640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.295+0000 7f0aecb58640 10 monclient: monmap: 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:54.296746+0000 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:54.296746+0000 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.292 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.293 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 1 Processor -- start 2024-05-08T15:46:54.300 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 1 -- start start 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient: init 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient: _add_conns ranks=[0,6,1,3,5,2,8,4,7] 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient(hunting): picked mon.noname-a con 0x7f0ae80a1390 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient(hunting): picked mon.noname-d con 0x7f0ae80a1770 addr v1:172.21.15.7:6792/0 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient(hunting): picked mon.noname-f con 0x7f0ae80a1b70 addr v1:172.21.15.81:6789/0 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae8136370 con 0x7f0ae80a1390 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae8072650 con 0x7f0ae80a1b70 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae8071550 con 0x7f0ae80a1770 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.296+0000 7f0aecb58640 10 monclient(hunting): authenticate will time out at 805.517253s 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae6d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0ae80a1b70 0x7f0ae813dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47254/0 (socket says 172.21.15.81:47254) 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae6d9d640 1 -- 172.21.15.81:0/2129099861 learned_addr learned my addr 172.21.15.81:0/2129099861 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:54.301 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049540160 0 0) 0x7f0ae0001490 con 0x7f0ae80a1b70 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient(hunting): my global_id is 4132 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ad4003520 con 0x7f0ae80a1b70 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1080154467 0 0) 0x7f0ad4003520 con 0x7f0ae80a1b70 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 >> v1:172.21.15.7:6792/0 conn(0x7f0ae80a1770 legacy=0x7f0ae813a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 >> v1:172.21.15.7:6789/0 conn(0x7f0ae80a1390 legacy=0x7f0ae8135c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 monclient: found mon.noname-f 2024-05-08T15:46:54.302 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae8072930 con 0x7f0ae80a1b70 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0aecb58640 5 monclient: authenticate success, global_id 4132 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0aecb58640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ae0001d00 con 0x7f0ae80a1b70 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: dump: 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.303 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0aecb58640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0ae0002060 con 0x7f0ae80a1b70 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.304 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 1 -- 172.21.15.81:0/2129099861 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ae0004fe0 con 0x7f0ae80a1b70 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0ae5d9b640 10 monclient: dump: 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.305 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0aecb58640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.297+0000 7f0aecb58640 4 set_mon_vals no callback set 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 10 monclient: shutdown 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 1 -- 172.21.15.81:0/2129099861 >> v1:172.21.15.81:6789/0 conn(0x7f0ae80a1b70 legacy=0x7f0ae813dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 1 -- 172.21.15.81:0/2129099861 shutdown_connections 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 1 -- 172.21.15.81:0/2129099861 >> 172.21.15.81:0/2129099861 conn(0x7f0ae80a8360 msgr2=0x7f0ae80a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 1 -- 172.21.15.81:0/2129099861 shutdown_connections 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 1 -- 172.21.15.81:0/2129099861 wait complete. 2024-05-08T15:46:54.306 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.298+0000 7f0aecb58640 10 monclient: monmap: 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 0.000000 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:created 0.000000 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 1 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.307 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 1 Processor -- start 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 1 -- start start 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient: init 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient: _add_conns ranks=[8,6,7,1,0,3,2,5,4] 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient(hunting): picked mon.noname-e con 0x7f0ae80a1390 addr v1:172.21.15.7:6793/0 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient(hunting): picked mon.noname-d con 0x7f0ae80a1770 addr v1:172.21.15.7:6792/0 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient(hunting): picked mon.noname-i con 0x7f0ae80a1b70 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae8130c20 con 0x7f0ae80a1770 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae8130210 con 0x7f0ae80a1b70 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae8130940 con 0x7f0ae80a1390 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0aecb58640 10 monclient(hunting): authenticate will time out at 805.520044s 2024-05-08T15:46:54.308 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0ae6d9d640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7f0ae80a1b70 0x7f0ae81d3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.81:54442/0 (socket says 172.21.15.81:54442) 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.299+0000 7f0ae6d9d640 1 -- 172.21.15.81:0/1885127612 learned_addr learned my addr 172.21.15.81:0/1885127612 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590771508 0 0) 0x7f0ae8130210 con 0x7f0ae80a1b70 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): my global_id is 4147 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac8003520 con 0x7f0ae80a1b70 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3207315200 0 0) 0x7f0ae8130c20 con 0x7f0ae80a1770 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): my global_id is 74121 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ae8130210 con 0x7f0ae80a1770 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 655755874 0 0) 0x7f0ae8130940 con 0x7f0ae80a1390 2024-05-08T15:46:54.309 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): my global_id is 4157 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ae8130c20 con 0x7f0ae80a1390 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1954365296 0 0) 0x7f0ac8003520 con 0x7f0ae80a1b70 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ae8130940 con 0x7f0ae80a1b70 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ae0001de0 con 0x7f0ae80a1b70 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): dump: 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.310 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 660345766 0 0) 0x7f0ae8130210 con 0x7f0ae80a1770 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ac8003520 con 0x7f0ae80a1770 2024-05-08T15:46:54.311 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ad00019d0 con 0x7f0ae80a1770 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): dump: 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.312 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1519442459 0 0) 0x7f0ae8130c20 con 0x7f0ae80a1390 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ae8130210 con 0x7f0ae80a1390 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ad8001900 con 0x7f0ae80a1390 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient(hunting): dump: 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.313 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1203440163 0 0) 0x7f0ae8130940 con 0x7f0ae80a1b70 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 >> v1:172.21.15.7:6793/0 conn(0x7f0ae80a1390 legacy=0x7f0ae8128660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 >> v1:172.21.15.7:6792/0 conn(0x7f0ae80a1770 legacy=0x7f0ae81cfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.314 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 monclient: found mon.i 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae809e620 con 0x7f0ae80a1b70 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0aecb58640 5 monclient: authenticate success, global_id 4147 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0aecb58640 10 monclient: _renew_subs 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0aecb58640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae809cd10 con 0x7f0ae80a1b70 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0aecb58640 10 monclient: _renew_subs 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0aecb58640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f0ae8128d70 con 0x7f0ae80a1b70 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0ae0002060 con 0x7f0ae80a1b70 2024-05-08T15:46:54.315 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ae00016c0 con 0x7f0ae80a1b70 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0ae559a640 4 set_mon_vals no callback set 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 10 monclient: dump: 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.317 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.317 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.317 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.317 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.317 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.317 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f0ae00058a0 con 0x7f0ae80a1b70 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.300+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (unknown 1111159847 0 0) 0x7f0ae0007110 con 0x7f0ae80a1b70 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.301+0000 7f0aecb58640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.301+0000 7f0aecb58640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.301+0000 7f0aecb58640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.301+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0acc005080 con 0x7f0ae80a1b70 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.303+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f0ae8072650 con 0x7f0ae80a1b70 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.303+0000 7f0adeffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.303+0000 7f0adeffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:54.363 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:54.367+0000 7fcfa3c21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T15:46:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.374+0000 7f0aecb58640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "64485986-c4f5-4899-8a0c-604433e57b46", "id": 4}] 2024-05-08T15:46:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.374+0000 7f0aecb58640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "64485986-c4f5-4899-8a0c-604433e57b46", "id": 4}] 2024-05-08T15:46:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.374+0000 7f0aecb58640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:46:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.374+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "osd new", "uuid": "64485986-c4f5-4899-8a0c-604433e57b46", "id": 4} v 0) -- 0x7f0acc005370 con 0x7f0ae80a1b70 2024-05-08T15:46:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.383+0000 7f0adeffd640 1 -- 172.21.15.81:0/1885127612 <== mon.7 v1:172.21.15.81:6792/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64485986-c4f5-4899-8a0c-604433e57b46", "id": 4}]=0 v7) ==== 112+0+2 (unknown 127913791 0 1697012583) 0x7f0ae0006d80 con 0x7f0ae80a1b70 2024-05-08T15:46:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.383+0000 7f0adeffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "64485986-c4f5-4899-8a0c-604433e57b46", "id": 4}] 2024-05-08T15:46:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.383+0000 7f0adeffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:54.380 INFO:teuthology.orchestra.run.smithi081.stdout:4 2024-05-08T15:46:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.384+0000 7f0aecb58640 10 monclient: shutdown 2024-05-08T15:46:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.384+0000 7f0aecb58640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.384+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 >> v1:172.21.15.81:6792/0 conn(0x7f0ae80a1b70 legacy=0x7f0ae81d3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.384+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 shutdown_connections 2024-05-08T15:46:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.384+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 >> 172.21.15.81:0/1885127612 conn(0x7f0ae80a8360 msgr2=0x7f0ae812eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.384+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 shutdown_connections 2024-05-08T15:46:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.384+0000 7f0aecb58640 1 -- 172.21.15.81:0/1885127612 wait complete. 2024-05-08T15:46:54.391 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 9a6e6150-5226-49ae-8598-7cf36b0f42d4 5 2024-05-08T15:46:54.457 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:54.460+0000 7fcfa3c21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T15:46:54.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.492+0000 7fa28a638640 10 monclient: get_monmap_and_config 2024-05-08T15:46:54.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.492+0000 7fa28a638640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.488 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.492+0000 7fa28a638640 10 monclient: monmap: 2024-05-08T15:46:54.488 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:54.493092+0000 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:54.493092+0000 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.489 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.492+0000 7fa28a638640 1 Processor -- start 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.492+0000 7fa28a638640 1 -- start start 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.492+0000 7fa28a638640 10 monclient: init 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 10 monclient: _add_conns ranks=[2,1,5,8,3,0,4,7,6] 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 10 monclient(hunting): picked mon.noname-b con 0x7fa2840a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 10 monclient(hunting): picked mon.noname-f con 0x7fa2840a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 10 monclient(hunting): picked mon.noname-h con 0x7fa28413e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840a1660 con 0x7fa2840a5780 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840a28f0 con 0x7fa2840a53a0 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840a2bd0 con 0x7fa28413e310 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa28a638640 10 monclient(hunting): authenticate will time out at 805.713719s 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa2837fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa2840a5780 0x7fa28413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47264/0 (socket says 172.21.15.81:47264) 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa2837fe640 1 -- 172.21.15.81:0/936162120 learned_addr learned my addr 172.21.15.81:0/936162120 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa288bd5640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fa28413e310 0x7fa284140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.81:60084/0 (socket says 172.21.15.81:60084) 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1550718778 0 0) 0x7fa2840a2bd0 con 0x7fa28413e310 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): my global_id is 4127 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa26c003520 con 0x7fa28413e310 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4161804653 0 0) 0x7fa2840a1660 con 0x7fa2840a5780 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 10 monclient(hunting): my global_id is 4141 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2840a2bd0 con 0x7fa2840a5780 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3672917197 0 0) 0x7fa26c003520 con 0x7fa28413e310 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.493+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 >> v1:172.21.15.7:6790/0 conn(0x7fa2840a53a0 legacy=0x7fa2840a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 >> v1:172.21.15.81:6789/0 conn(0x7fa2840a5780 legacy=0x7fa28413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 1 monclient: found mon.noname-h 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2840a6730 con 0x7fa28413e310 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa27c001b30 con 0x7fa28413e310 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: dump: 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 5 monclient: authenticate success, global_id 4127 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa27c001e10 con 0x7fa28413e310 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 1 -- 172.21.15.81:0/936162120 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa27c004ea0 con 0x7fa28413e310 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 4 set_mon_vals no callback set 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa282ffd640 10 monclient: dump: 2024-05-08T15:46:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 10 monclient: shutdown 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 1 -- 172.21.15.81:0/936162120 >> v1:172.21.15.81:6791/0 conn(0x7fa28413e310 legacy=0x7fa284140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 1 -- 172.21.15.81:0/936162120 shutdown_connections 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 1 -- 172.21.15.81:0/936162120 >> 172.21.15.81:0/936162120 conn(0x7fa2840a9fa0 msgr2=0x7fa2840aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 1 -- 172.21.15.81:0/936162120 shutdown_connections 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.494+0000 7fa28a638640 1 -- 172.21.15.81:0/936162120 wait complete. 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.495+0000 7fa28a638640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.495+0000 7fa28a638640 10 monclient: monmap: 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 0 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 0.000000 2024-05-08T15:46:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:created 0.000000 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 1 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.495+0000 7fa28a638640 1 Processor -- start 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.495+0000 7fa28a638640 1 -- start start 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.495+0000 7fa28a638640 10 monclient: init 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient: _add_conns ranks=[0,5,4,7,8,2,6,3,1] 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient(hunting): picked mon.noname-a con 0x7fa2840a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient(hunting): picked mon.noname-h con 0x7fa2840a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient(hunting): picked mon.noname-c con 0x7fa28413e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2841cdb30 con 0x7fa2840a53a0 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840a1e00 con 0x7fa28413e310 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2840a43c0 con 0x7fa2840a5780 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient(hunting): authenticate will time out at 805.716510s 2024-05-08T15:46:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa2837fe640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fa2840a5780 0x7fa2841cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.81:60088/0 (socket says 172.21.15.81:60088) 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa2837fe640 1 -- 172.21.15.81:0/1617768460 learned_addr learned my addr 172.21.15.81:0/1617768460 (peer_addr_for_me v1:172.21.15.81:0/0) 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 157686371 0 0) 0x7fa2840a43c0 con 0x7fa2840a5780 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): my global_id is 4136 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa270003520 con 0x7fa2840a5780 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897229668 0 0) 0x7fa2840a1e00 con 0x7fa28413e310 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): my global_id is 4135 2024-05-08T15:46:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2840a43c0 con 0x7fa28413e310 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3682306712 0 0) 0x7fa2841cdb30 con 0x7fa2840a53a0 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): my global_id is 4158 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2840a1e00 con 0x7fa2840a53a0 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1097085352 0 0) 0x7fa270003520 con 0x7fa2840a5780 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa2841cdb30 con 0x7fa2840a5780 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa278001960 con 0x7fa2840a5780 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): dump: 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4196469438 0 0) 0x7fa2841cdb30 con 0x7fa2840a5780 2024-05-08T15:46:54.508 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 >> v1:172.21.15.7:6791/0 conn(0x7fa28413e310 legacy=0x7fa2841d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 >> v1:172.21.15.7:6789/0 conn(0x7fa2840a53a0 legacy=0x7fa2841ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 monclient: found mon.h 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa284134550 con 0x7fa2840a5780 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa280ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 5 monclient: authenticate success, global_id 4136 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient: _renew_subs 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.496+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.81:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa284134830 con 0x7fa2840a5780 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa28a638640 10 monclient: _renew_subs 2024-05-08T15:46:54.509 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa28a638640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.81:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fa284132f30 con 0x7fa2840a5780 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa2780047e0 con 0x7fa2840a5780 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa278004cd0 con 0x7fa2840a5780 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa2827fc640 4 set_mon_vals no callback set 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 10 monclient: dump: 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:epoch 1 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:election_strategy: 3 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7fa2780052e0 con 0x7fa2840a5780 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.497+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 8 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (unknown 2484199174 0 0) 0x7fa2780055d0 con 0x7fa2840a5780 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.498+0000 7fa28a638640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.498+0000 7fa28a638640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.498+0000 7fa28a638640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.498+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa264005080 con 0x7fa2840a5780 2024-05-08T15:46:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.500+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fa278006020 con 0x7fa2840a5780 2024-05-08T15:46:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.500+0000 7fa280ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.500+0000 7fa280ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:54.512 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:54.505+0000 7fcfa3c21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T15:46:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.571+0000 7fa28a638640 10 start_mon_command cmd=[{"prefix": "osd new", "uuid": "9a6e6150-5226-49ae-8598-7cf36b0f42d4", "id": 5}] 2024-05-08T15:46:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.571+0000 7fa28a638640 10 monclient: _send_command 2 [{"prefix": "osd new", "uuid": "9a6e6150-5226-49ae-8598-7cf36b0f42d4", "id": 5}] 2024-05-08T15:46:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.571+0000 7fa28a638640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.568 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.571+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "osd new", "uuid": "9a6e6150-5226-49ae-8598-7cf36b0f42d4", "id": 5} v 0) -- 0x7fa264005370 con 0x7fa2840a5780 2024-05-08T15:46:54.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.580+0000 7fa280ff9640 1 -- 172.21.15.81:0/1617768460 <== mon.5 v1:172.21.15.81:6791/0 10 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a6e6150-5226-49ae-8598-7cf36b0f42d4", "id": 5}]=0 v8) ==== 112+0+2 (unknown 1133975473 0 1988412176) 0x7fa2780059a0 con 0x7fa2840a5780 2024-05-08T15:46:54.576 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.580+0000 7fa280ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd new", "uuid": "9a6e6150-5226-49ae-8598-7cf36b0f42d4", "id": 5}] 2024-05-08T15:46:54.577 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.580+0000 7fa280ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:54.577 INFO:teuthology.orchestra.run.smithi081.stdout:5 2024-05-08T15:46:54.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.581+0000 7fa28a638640 10 monclient: shutdown 2024-05-08T15:46:54.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.581+0000 7fa28a638640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.581+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 >> v1:172.21.15.81:6791/0 conn(0x7fa2840a5780 legacy=0x7fa2841cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.581+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 shutdown_connections 2024-05-08T15:46:54.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.581+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 >> 172.21.15.81:0/1617768460 conn(0x7fa2840a9fa0 msgr2=0x7fa28413eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.581+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 shutdown_connections 2024-05-08T15:46:54.578 INFO:teuthology.orchestra.run.smithi081.stderr:2024-05-08T15:46:54.581+0000 7fa28a638640 1 -- 172.21.15.81:0/1617768460 wait complete. 2024-05-08T15:46:54.589 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T15:46:54.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T15:46:54.592 INFO:tasks.ceph.osd.0:Started 2024-05-08T15:46:54.592 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T15:46:54.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T15:46:54.594 INFO:tasks.ceph.osd.1:Started 2024-05-08T15:46:54.595 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T15:46:54.595 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T15:46:54.597 INFO:tasks.ceph.osd.2:Started 2024-05-08T15:46:54.597 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T15:46:54.597 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T15:46:54.600 INFO:tasks.ceph.osd.3:Started 2024-05-08T15:46:54.600 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T15:46:54.600 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T15:46:54.602 INFO:tasks.ceph.osd.4:Started 2024-05-08T15:46:54.602 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T15:46:54.602 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T15:46:54.604 INFO:tasks.ceph.osd.5:Started 2024-05-08T15:46:54.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T15:46:54.682 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:54.686+0000 7fcfa3c21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.726+0000 7ffaf6fe6640 10 monclient: get_monmap_and_config 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.726+0000 7ffaf6fe6640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.726+0000 7ffaf6fe6640 10 monclient: monmap: 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:54.727628+0000 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:54.727628+0000 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.723 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.724 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.724 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.727+0000 7ffaf6fe6640 1 Processor -- start 2024-05-08T15:46:54.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.727+0000 7ffaf6fe6640 1 -- start start 2024-05-08T15:46:54.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.727+0000 7ffaf6fe6640 10 monclient: init 2024-05-08T15:46:54.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.727+0000 7ffaf6fe6640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.727+0000 7ffaf6fe6640 10 monclient: _add_conns ranks=[8,5,6,1,4,2,7,0,3] 2024-05-08T15:46:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.727+0000 7ffaf6fe6640 10 monclient(hunting): picked mon.noname-e con 0x7ffaf0137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:46:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.727+0000 7ffaf6fe6640 10 monclient(hunting): picked mon.noname-h con 0x7ffaf00a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:46:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 10 monclient(hunting): picked mon.noname-d con 0x7ffaf00a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:46:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf00a2c70 con 0x7ffaf00a49a0 2024-05-08T15:46:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf00a2f50 con 0x7ffaf00a41b0 2024-05-08T15:46:54.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf00a1670 con 0x7ffaf0137940 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 10 monclient(hunting): authenticate will time out at 822.842699s 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf4d82640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7ffaf0137940 0x7ffaf0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:40040/0 (socket says 172.21.15.7:40040) 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf5583640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7ffaf00a41b0 0x7ffaf0140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:59594/0 (socket says 172.21.15.7:59594) 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf4d82640 1 -- 172.21.15.7:0/3670150044 learned_addr learned my addr 172.21.15.7:0/3670150044 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1793579149 0 0) 0x7ffaf00a2f50 con 0x7ffaf00a41b0 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): my global_id is 74148 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffae0003520 con 0x7ffaf00a41b0 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 208612270 0 0) 0x7ffaf00a1670 con 0x7ffaf0137940 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): my global_id is 4175 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaf00a2f50 con 0x7ffaf0137940 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3065319559 0 0) 0x7ffaf00a2c70 con 0x7ffaf00a49a0 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient(hunting): my global_id is 4163 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaf00a1670 con 0x7ffaf00a49a0 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3185802815 0 0) 0x7ffaf00a1670 con 0x7ffaf00a49a0 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 >> v1:172.21.15.7:6793/0 conn(0x7ffaf0137940 legacy=0x7ffaf0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 >> v1:172.21.15.7:6792/0 conn(0x7ffaf00a41b0 legacy=0x7ffaf0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 monclient: found mon.noname-h 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf00a39f0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffadc001750 con 0x7ffaf00a49a0 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: dump: 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.737 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 5 monclient: authenticate success, global_id 4163 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffadc0019b0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 1 -- 172.21.15.7:0/3670150044 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffadc003950 con 0x7ffaf00a49a0 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:46:54.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaef7fe640 10 monclient: dump: 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.739 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.728+0000 7ffaf6fe6640 4 set_mon_vals no callback set 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.729+0000 7ffaf6fe6640 10 monclient: shutdown 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.729+0000 7ffaf6fe6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.729+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/3670150044 >> v1:172.21.15.81:6791/0 conn(0x7ffaf00a49a0 legacy=0x7ffaf013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.729+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/3670150044 shutdown_connections 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.729+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/3670150044 >> 172.21.15.7:0/3670150044 conn(0x7ffaf00a9fa0 msgr2=0x7ffaf00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.729+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/3670150044 shutdown_connections 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.729+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/3670150044 wait complete. 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.730+0000 7ffaf6fe6640 10 monclient: build_initial_monmap 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.730+0000 7ffaf6fe6640 10 monclient: monmap: 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:54.741 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.730+0000 7ffaf6fe6640 1 Processor -- start 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.730+0000 7ffaf6fe6640 1 -- start start 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.730+0000 7ffaf6fe6640 10 monclient: init 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.730+0000 7ffaf6fe6640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.730+0000 7ffaf6fe6640 10 monclient: _add_conns ranks=[7,5,3,0,4,6,2,8,1] 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 10 monclient(hunting): picked mon.noname-i con 0x7ffaf00a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 10 monclient(hunting): picked mon.noname-h con 0x7ffaf00a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:46:54.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 10 monclient(hunting): picked mon.noname-g con 0x7ffaf0137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf00a1f30 con 0x7ffaf0137940 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf0135890 con 0x7ffaf00a49a0 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaf01cceb0 con 0x7ffaf00a41b0 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 10 monclient(hunting): authenticate will time out at 822.845708s 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaeffff640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7ffaf00a49a0 0x7ffaf01d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:35990/0 (socket says 172.21.15.7:35990) 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaeffff640 1 -- 172.21.15.7:0/2649821758 learned_addr learned my addr 172.21.15.7:0/2649821758 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf4d82640 1 --1- 172.21.15.7:0/2649821758 >> v1:172.21.15.81:6792/0 conn(0x7ffaf00a41b0 0x7ffaf0136dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:36312/0 (socket says 172.21.15.7:36312) 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364280992 0 0) 0x7ffaf0135890 con 0x7ffaf00a49a0 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): my global_id is 4172 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffad4003550 con 0x7ffaf00a49a0 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136354070 0 0) 0x7ffaf01cceb0 con 0x7ffaf00a41b0 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): my global_id is 4174 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaf0135890 con 0x7ffaf00a41b0 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367469708 0 0) 0x7ffae0003520 con 0x7ffaf0137940 2024-05-08T15:46:54.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): my global_id is 74109 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffaf01cceb0 con 0x7ffaf0137940 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2971619758 0 0) 0x7ffad4003550 con 0x7ffaf00a49a0 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffae0003520 con 0x7ffaf00a49a0 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffadc001690 con 0x7ffaf00a49a0 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): dump: 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.745 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:54.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2743169355 0 0) 0x7ffaf0135890 con 0x7ffaf00a41b0 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffad4003550 con 0x7ffaf00a41b0 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffae4001970 con 0x7ffaf00a41b0 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): dump: 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.747 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 240917412 0 0) 0x7ffaf01cceb0 con 0x7ffaf0137940 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffaf0135890 con 0x7ffaf0137940 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffae8001ca0 con 0x7ffaf0137940 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient(hunting): dump: 2024-05-08T15:46:54.748 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:54.749 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.749 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.749 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.749 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.749 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:54.749 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.749 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.750 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.750 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.750 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.750 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.750 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.750 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.750 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3512560403 0 0) 0x7ffae0003520 con 0x7ffaf00a49a0 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 >> v1:172.21.15.81:6792/0 conn(0x7ffaf00a41b0 legacy=0x7ffaf0136dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 >> v1:172.21.15.81:6790/0 conn(0x7ffaf0137940 legacy=0x7ffaf01d4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 monclient: found mon.h 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf0134ff0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient: _finish_auth 0 2024-05-08T15:46:54.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaed7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 5 monclient: authenticate success, global_id 4172 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 10 monclient: _renew_subs 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.731+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf01352d0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaf6fe6640 10 monclient: _renew_subs 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaf6fe6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7ffaf01374d0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffadc0019b0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffadc004250 con 0x7ffaf00a49a0 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:46:54.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaeeffd640 4 set_mon_vals no callback set 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 10 monclient: dump: 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:54.753 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7ffadc004880 con 0x7ffaf00a49a0 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.732+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 8 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (unknown 2153706066 0 0) 0x7ffadc0063e0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.733+0000 7ffaf6fe6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.733+0000 7ffaf6fe6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.733+0000 7ffaf6fe6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.733+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffad0005080 con 0x7ffaf00a49a0 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.735+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7ffadc001c20 con 0x7ffaf00a49a0 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.735+0000 7ffaed7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:54.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.735+0000 7ffaed7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:54.755 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:54.737+0000 7fcfa3c21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T15:46:54.786 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:54.789+0000 7fcfa3c21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T15:46:54.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.797+0000 7ffaf6fe6640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:46:54.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.797+0000 7ffaf6fe6640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:46:54.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.797+0000 7ffaf6fe6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:46:54.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.797+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffad0005370 con 0x7ffaf00a49a0 2024-05-08T15:46:54.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.798+0000 7ffaed7fa640 1 -- 172.21.15.7:0/2649821758 <== mon.5 v1:172.21.15.81:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (unknown 150273500 0 771082721) 0x7ffadc005df0 con 0x7ffaf00a49a0 2024-05-08T15:46:54.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.798+0000 7ffaed7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:46:54.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.798+0000 7ffaed7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:54.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:46:54.795 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:46:54.573248+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:46:54.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.798+0000 7ffaf6fe6640 10 monclient: shutdown 2024-05-08T15:46:54.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.798+0000 7ffaf6fe6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:54.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.798+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 >> v1:172.21.15.81:6791/0 conn(0x7ffaf00a49a0 legacy=0x7ffaf01d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:54.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.799+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 shutdown_connections 2024-05-08T15:46:54.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.799+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 >> 172.21.15.7:0/2649821758 conn(0x7ffaf00a9fa0 msgr2=0x7ffaf00a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:54.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.799+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 shutdown_connections 2024-05-08T15:46:54.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:46:54.799+0000 7ffaf6fe6640 1 -- 172.21.15.7:0/2649821758 wait complete. 2024-05-08T15:46:54.806 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T15:46:54.806 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T15:46:54.897 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:54.901+0000 7fcfa3c21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T15:46:55.199 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-05-08T15:46:55.202+0000 7f85ffd68640 -1 Falling back to public interface 2024-05-08T15:46:55.199 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T15:46:55.202+0000 7f55f94b3640 -1 Falling back to public interface 2024-05-08T15:46:55.199 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T15:46:55.202+0000 7f55f8462640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2024-05-08T15:46:55.200 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T15:46:55.203+0000 7f55f8462640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6804/0: (98) Address already in use 2024-05-08T15:46:55.200 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-05-08T15:46:55.203+0000 7f0046525640 -1 Falling back to public interface 2024-05-08T15:46:55.202 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-05-08T15:46:55.205+0000 7fc53f20e640 -1 Falling back to public interface 2024-05-08T15:46:55.202 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-05-08T15:46:55.206+0000 7fd0035b6640 -1 Falling back to public interface 2024-05-08T15:46:55.208 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-05-08T15:46:55.212+0000 7f3606f0a640 -1 Falling back to public interface 2024-05-08T15:46:55.279 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:55.283+0000 7fcfa3c21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T15:46:55.410 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T15:46:55.511 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.515+0000 7f3d8a99f640 10 monclient: get_monmap_and_config 2024-05-08T15:46:55.512 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.515+0000 7f3d8a99f640 10 monclient: build_initial_monmap 2024-05-08T15:46:55.512 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.515+0000 7f3d8a99f640 10 monclient: monmap: 2024-05-08T15:46:55.512 INFO:teuthology.misc.health.smithi007.stderr:epoch 0 2024-05-08T15:46:55.512 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:55.512 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:55.516071+0000 2024-05-08T15:46:55.512 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:55.516071+0000 2024-05-08T15:46:55.513 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:55.513 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:55.513 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:55.513 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:55.513 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:55.513 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:55.514 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:55.514 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:55.514 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:55.514 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:55.514 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:55.514 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:46:55.530 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.515+0000 7f3d8a99f640 1 Processor -- start 2024-05-08T15:46:55.530 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 1 -- start start 2024-05-08T15:46:55.531 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient: init 2024-05-08T15:46:55.531 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:55.531 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient: _add_conns ranks=[2,7,0,6,8,5,1,4,3] 2024-05-08T15:46:55.531 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient(hunting): picked mon.noname-b con 0x7f3d840a3d40 addr v1:172.21.15.7:6790/0 2024-05-08T15:46:55.531 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient(hunting): picked mon.noname-i con 0x7f3d8409f640 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:55.531 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient(hunting): picked mon.noname-a con 0x7f3d8413de30 addr v1:172.21.15.7:6789/0 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d841387b0 con 0x7f3d8413de30 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d840a0af0 con 0x7f3d840a3d40 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d840a75c0 con 0x7f3d8409f640 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a99f640 10 monclient(hunting): authenticate will time out at 823.631194s 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8999d640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f3d840a3d40 0x7f3d840a4120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35266/0 (socket says 172.21.15.7:35266) 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8999d640 1 -- 172.21.15.7:0/269766047 learned_addr learned my addr 172.21.15.7:0/269766047 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8a19e640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3d8413de30 0x7f3d84140220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47318/0 (socket says 172.21.15.7:47318) 2024-05-08T15:46:55.532 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1098277155 0 0) 0x7f3d841387b0 con 0x7f3d8413de30 2024-05-08T15:46:55.533 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8899b640 10 monclient(hunting): _negotiate 2024-05-08T15:46:55.533 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8899b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:55.533 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8899b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:55.533 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.516+0000 7f3d8899b640 10 monclient(hunting): my global_id is 4194 2024-05-08T15:46:55.533 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d70003520 con 0x7f3d8413de30 2024-05-08T15:46:55.533 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209470411 0 0) 0x7f3d840a0af0 con 0x7f3d840a3d40 2024-05-08T15:46:55.533 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): _negotiate 2024-05-08T15:46:55.534 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:55.534 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:55.534 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): my global_id is 74117 2024-05-08T15:46:55.534 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d841387b0 con 0x7f3d840a3d40 2024-05-08T15:46:55.534 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825550044 0 0) 0x7f3d840a75c0 con 0x7f3d8409f640 2024-05-08T15:46:55.534 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): _negotiate 2024-05-08T15:46:55.534 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:55.535 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:55.535 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient(hunting): my global_id is 4183 2024-05-08T15:46:55.535 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d840a0af0 con 0x7f3d8409f640 2024-05-08T15:46:55.535 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3126115148 0 0) 0x7f3d70003520 con 0x7f3d8413de30 2024-05-08T15:46:55.535 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 >> v1:172.21.15.81:6792/0 conn(0x7f3d8409f640 legacy=0x7f3d841380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:55.535 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 >> v1:172.21.15.7:6790/0 conn(0x7f3d840a3d40 legacy=0x7f3d840a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:55.535 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: _finish_hunting 0 2024-05-08T15:46:55.536 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 monclient: found mon.noname-a 2024-05-08T15:46:55.536 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:55.536 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:46:55.536 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d840a7790 con 0x7f3d8413de30 2024-05-08T15:46:55.536 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: _finish_auth 0 2024-05-08T15:46:55.536 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: _check_auth_tickets 2024-05-08T15:46:55.537 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:55.537 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3d7c001b30 con 0x7f3d8413de30 2024-05-08T15:46:55.537 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 5 monclient: authenticate success, global_id 4194 2024-05-08T15:46:55.537 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:55.537 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:55.537 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:46:55.537 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: dump: 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:55.538 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3d7c001d90 con 0x7f3d8413de30 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 1 -- 172.21.15.7:0/269766047 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3d7c004ea0 con 0x7f3d8413de30 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 10 monclient: get_monmap_and_config success 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 4 set_mon_vals no callback set 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:55.539 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8899b640 10 monclient: dump: 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:55.540 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 10 monclient: shutdown 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 1 -- 172.21.15.7:0/269766047 >> v1:172.21.15.7:6789/0 conn(0x7f3d8413de30 legacy=0x7f3d84140220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 1 -- 172.21.15.7:0/269766047 shutdown_connections 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.517+0000 7f3d8a99f640 1 -- 172.21.15.7:0/269766047 >> 172.21.15.7:0/269766047 conn(0x7f3d840aa830 msgr2=0x7f3d840a6cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.518+0000 7f3d8a99f640 1 -- 172.21.15.7:0/269766047 shutdown_connections 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.518+0000 7f3d8a99f640 1 -- 172.21.15.7:0/269766047 wait complete. 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.518+0000 7f3d8a99f640 10 monclient: build_initial_monmap 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.518+0000 7f3d8a99f640 10 monclient: monmap: 2024-05-08T15:46:55.541 INFO:teuthology.misc.health.smithi007.stderr:epoch 0 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:last_changed 0.000000 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:created 0.000000 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 1 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:46:55.542 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.518+0000 7f3d8a99f640 1 Processor -- start 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.518+0000 7f3d8a99f640 1 -- start start 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.518+0000 7f3d8a99f640 10 monclient: init 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 10 monclient: _reopen_session rank -1 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 10 monclient: _add_conns ranks=[6,3,7,8,0,1,5,4,2] 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 10 monclient(hunting): picked mon.noname-d con 0x7f3d8409f640 addr v1:172.21.15.7:6792/0 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 10 monclient(hunting): picked mon.noname-g con 0x7f3d840a3d40 addr v1:172.21.15.81:6790/0 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 10 monclient(hunting): picked mon.noname-i con 0x7f3d8413de30 addr v1:172.21.15.81:6792/0 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d84139060 con 0x7f3d840a3d40 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d84138c40 con 0x7f3d8409f640 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d84070a30 con 0x7f3d8413de30 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 10 monclient(hunting): _renew_subs 2024-05-08T15:46:55.543 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8a99f640 10 monclient(hunting): authenticate will time out at 823.634042s 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8999d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f3d8409f640 0x7f3d8406e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:59606/0 (socket says 172.21.15.7:59606) 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d8999d640 1 -- 172.21.15.7:0/2276907289 learned_addr learned my addr 172.21.15.7:0/2276907289 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463583106 0 0) 0x7f3d84138c40 con 0x7f3d8409f640 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): my global_id is 74157 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d68003520 con 0x7f3d8409f640 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684773934 0 0) 0x7f3d84070a30 con 0x7f3d8413de30 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): my global_id is 4192 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d84138c40 con 0x7f3d8413de30 2024-05-08T15:46:55.544 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991181964 0 0) 0x7f3d74001a90 con 0x7f3d840a3d40 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): my global_id is 74118 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d84070a30 con 0x7f3d840a3d40 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2050988003 0 0) 0x7f3d68003520 con 0x7f3d8409f640 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d74001a90 con 0x7f3d8409f640 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3d80001500 con 0x7f3d8409f640 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): dump: 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.545 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.519+0000 7f3d7a7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3602806208 0 0) 0x7f3d84138c40 con 0x7f3d8413de30 2024-05-08T15:46:55.546 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3d68003520 con 0x7f3d8413de30 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3d7c001a30 con 0x7f3d8413de30 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient(hunting): dump: 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:55.547 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2441973634 0 0) 0x7f3d74001a90 con 0x7f3d8409f640 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 >> v1:172.21.15.81:6792/0 conn(0x7f3d8413de30 legacy=0x7f3d841d4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 >> v1:172.21.15.81:6790/0 conn(0x7f3d840a3d40 legacy=0x7f3d8406ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 monclient: found mon.d 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:46:55.548 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d84070c50 con 0x7f3d8409f640 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d8a99f640 5 monclient: authenticate success, global_id 74157 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d8a99f640 10 monclient: _renew_subs 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d8a99f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3d84072590 con 0x7f3d8409f640 2024-05-08T15:46:55.549 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d8a99f640 10 monclient: _renew_subs 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d8a99f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f3d84071c00 con 0x7f3d8409f640 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3d80004a90 con 0x7f3d8409f640 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3d80004f80 con 0x7f3d8409f640 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 10 monclient: dump: 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:46:55.550 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:46:55.551 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 1) ==== 807+0+0 (unknown 3189685207 0 0) 0x7f3d800054b0 con 0x7f3d8409f640 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (unknown 2153706066 0 0) 0x7f3d80006da0 con 0x7f3d8409f640 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.520+0000 7f3d7bfff640 4 set_mon_vals no callback set 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.521+0000 7f3d8a99f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.521+0000 7f3d8a99f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.521+0000 7f3d8a99f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.521+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c005080 con 0x7f3d8409f640 2024-05-08T15:46:55.552 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.523+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f3d800067b0 con 0x7f3d8409f640 2024-05-08T15:46:55.553 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.523+0000 7f3d7a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:46:55.553 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.523+0000 7f3d7a7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:46:55.553 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:55.541+0000 7fcfa3c21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T15:46:55.582 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.585+0000 7f3d8a99f640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:46:55.582 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.585+0000 7f3d8a99f640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:46:55.582 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.585+0000 7f3d8a99f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:46:55.582 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.585+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d6c005370 con 0x7f3d8409f640 2024-05-08T15:46:55.582 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.585+0000 7f3d7a7fc640 1 -- 172.21.15.7:0/2276907289 <== mon.6 v1:172.21.15.7:6792/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) ==== 74+0+5605 (unknown 150273500 0 771082721) 0x7f3d80006aa0 con 0x7f3d8409f640 2024-05-08T15:46:55.582 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.585+0000 7f3d7a7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:46:55.582 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.585+0000 7f3d7a7fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:46:55.583 INFO:teuthology.misc.health.smithi007.stdout: 2024-05-08T15:46:55.583 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":8,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:46:54.573248+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:46:55.584 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.586+0000 7f3d8a99f640 10 monclient: shutdown 2024-05-08T15:46:55.584 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.586+0000 7f3d8a99f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:46:55.584 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.586+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 >> v1:172.21.15.7:6792/0 conn(0x7f3d8409f640 legacy=0x7f3d8406e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:46:55.584 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.586+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 shutdown_connections 2024-05-08T15:46:55.585 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.586+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 >> 172.21.15.7:0/2276907289 conn(0x7f3d840aa830 msgr2=0x7f3d840a8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:46:55.585 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.586+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 shutdown_connections 2024-05-08T15:46:55.585 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:46:55.586+0000 7f3d8a99f640 1 -- 172.21.15.7:0/2276907289 wait complete. 2024-05-08T15:46:55.588 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:55.592+0000 7fcfa3c21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T15:46:55.591 DEBUG:teuthology.misc:0 of 6 OSDs are up 2024-05-08T15:46:55.703 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:55.706+0000 7fcfa3c21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T15:46:55.997 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.001+0000 7fcfa3c21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T15:46:56.046 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.049+0000 7fcfa3c21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T15:46:56.201 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.204+0000 7fcfa3c21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T15:46:56.336 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.339+0000 7fcfa3c21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T15:46:56.385 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.388+0000 7fcfa3c21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T15:46:56.453 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.456+0000 7fcfa3c21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T15:46:56.497 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.501+0000 7fcfa3c21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T15:46:56.969 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:56.973+0000 7fcfa3c21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T15:46:57.082 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:57.086+0000 7fcfa3c21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T15:46:58.096 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.100+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.f 2024-05-08T15:46:58.097 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.100+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.g 2024-05-08T15:46:58.097 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.100+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.d 2024-05-08T15:46:58.097 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.101+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T15:46:58.097 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.101+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T15:46:58.097 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.101+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.e 2024-05-08T15:46:58.098 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.101+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.i 2024-05-08T15:46:58.098 INFO:tasks.ceph.mgr.x.smithi081.stderr:2024-05-08T15:46:58.101+0000 7fcf44450640 -1 mgr.server handle_report got status from non-daemon mon.h 2024-05-08T15:46:58.479 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-05-08T15:46:58.482+0000 7fc53f20e640 -1 osd.1 0 log_to_monitors true 2024-05-08T15:46:58.704 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-05-08T15:46:58.707+0000 7fd0035b6640 -1 osd.0 0 log_to_monitors true 2024-05-08T15:46:58.743 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-05-08T15:46:58.746+0000 7f3606f0a640 -1 osd.2 0 log_to_monitors true 2024-05-08T15:46:58.753 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-05-08T15:46:58.757+0000 7f85ffd68640 -1 osd.5 0 log_to_monitors true 2024-05-08T15:46:58.754 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T15:46:58.757+0000 7f55f94b3640 -1 osd.3 0 log_to_monitors true 2024-05-08T15:46:58.774 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-05-08T15:46:58.778+0000 7f0046525640 -1 osd.4 0 log_to_monitors true 2024-05-08T15:47:00.105 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-05-08T15:47:00.108+0000 7f00424bb640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T15:47:00.105 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T15:47:00.109+0000 7f55f5449640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T15:47:00.107 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-05-08T15:47:00.110+0000 7fc53b1a4640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T15:47:00.107 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-05-08T15:47:00.111+0000 7f85fbcfe640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T15:47:00.111 INFO:tasks.ceph.osd.3.smithi081.stderr:2024-05-08T15:47:00.115+0000 7f55f1271640 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T15:47:00.112 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-05-08T15:47:00.115+0000 7fc536fcc640 -1 osd.1 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T15:47:00.113 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-05-08T15:47:00.116+0000 7f3602ea0640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T15:47:00.113 INFO:tasks.ceph.osd.4.smithi081.stderr:2024-05-08T15:47:00.117+0000 7f003e2e3640 -1 osd.4 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T15:47:00.115 INFO:tasks.ceph.osd.5.smithi081.stderr:2024-05-08T15:47:00.118+0000 7f85f7b26640 -1 osd.5 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T15:47:00.117 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-05-08T15:47:00.120+0000 7f35fecc8640 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T15:47:02.198 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.300+0000 7f6e004d4640 10 monclient: get_monmap_and_config 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.300+0000 7f6e004d4640 10 monclient: build_initial_monmap 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.300+0000 7f6e004d4640 10 monclient: monmap: 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:epoch 0 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:47:02.301775+0000 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:47:02.301775+0000 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:02.297 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:02.298 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.306 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.301+0000 7f6e004d4640 1 Processor -- start 2024-05-08T15:47:02.306 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.301+0000 7f6e004d4640 1 -- start start 2024-05-08T15:47:02.306 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.301+0000 7f6e004d4640 10 monclient: init 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient: _add_conns ranks=[7,6,4,2,5,1,8,3,0] 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient(hunting): picked mon.noname-i con 0x7f6df809ea50 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient(hunting): picked mon.noname-d con 0x7f6df809e590 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient(hunting): picked mon.noname-c con 0x7f6df813e7d0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df80a0090 con 0x7f6df813e7d0 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df80a0650 con 0x7f6df809e590 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df80a0930 con 0x7f6df809ea50 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient(hunting): authenticate will time out at 830.416770s 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfda6f640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f6df809e590 0x7f6df813d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:59630/0 (socket says 172.21.15.7:59630) 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfda6f640 1 -- 172.21.15.7:0/3356840794 learned_addr learned my addr 172.21.15.7:0/3356840794 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:02.307 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfea71640 1 --1- 172.21.15.7:0/3356840794 >> v1:172.21.15.7:6791/0 conn(0x7f6df813e7d0 0x7f6df8140bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:44298/0 (socket says 172.21.15.7:44298) 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 764727050 0 0) 0x7f6df80a0650 con 0x7f6df809e590 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _negotiate 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): my global_id is 74202 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6de4003520 con 0x7f6df809e590 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1675945595 0 0) 0x7f6df80a0090 con 0x7f6df813e7d0 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _negotiate 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): my global_id is 4189 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df80a0650 con 0x7f6df813e7d0 2024-05-08T15:47:02.308 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 564763688 0 0) 0x7f6df80a0930 con 0x7f6df809ea50 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _negotiate 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient(hunting): my global_id is 4210 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df80a0090 con 0x7f6df809ea50 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2705143055 0 0) 0x7f6de4003520 con 0x7f6df809e590 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 >> v1:172.21.15.81:6792/0 conn(0x7f6df809ea50 legacy=0x7f6df809ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 >> v1:172.21.15.7:6791/0 conn(0x7f6df813e7d0 legacy=0x7f6df8140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: _finish_hunting 0 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 monclient: found mon.noname-d 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df80a61d0 con 0x7f6df809e590 2024-05-08T15:47:02.309 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: _finish_auth 0 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: _check_auth_tickets 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6df40017b0 con 0x7f6df809e590 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 5 monclient: authenticate success, global_id 74202 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: dump: 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:02.310 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6df4001a10 con 0x7f6df809e590 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6dfd26e640 1 -- 172.21.15.7:0/3356840794 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6df4004be0 con 0x7f6df809e590 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:02.311 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.302+0000 7f6e004d4640 4 set_mon_vals no callback set 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6dfd26e640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6dfd26e640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6dfd26e640 10 monclient: dump: 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:02.312 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6e004d4640 10 monclient: shutdown 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6e004d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6e004d4640 1 -- 172.21.15.7:0/3356840794 >> v1:172.21.15.7:6792/0 conn(0x7f6df809e590 legacy=0x7f6df813d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6e004d4640 1 -- 172.21.15.7:0/3356840794 shutdown_connections 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6e004d4640 1 -- 172.21.15.7:0/3356840794 >> 172.21.15.7:0/3356840794 conn(0x7f6df80a7020 msgr2=0x7f6df80a7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6e004d4640 1 -- 172.21.15.7:0/3356840794 shutdown_connections 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.303+0000 7f6e004d4640 1 -- 172.21.15.7:0/3356840794 wait complete. 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.304+0000 7f6e004d4640 10 monclient: build_initial_monmap 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.304+0000 7f6e004d4640 10 monclient: monmap: 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:epoch 0 2024-05-08T15:47:02.313 INFO:teuthology.misc.health.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:created 0.000000 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:02.314 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.304+0000 7f6e004d4640 1 Processor -- start 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.304+0000 7f6e004d4640 1 -- start start 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.304+0000 7f6e004d4640 10 monclient: init 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.304+0000 7f6e004d4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 10 monclient: _add_conns ranks=[2,3,4,8,1,6,7,5,0] 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 10 monclient(hunting): picked mon.noname-b con 0x7f6df809e590 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 10 monclient(hunting): picked mon.noname-g con 0x7f6df809ea50 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 10 monclient(hunting): picked mon.noname-c con 0x7f6df813e7d0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df809fbb0 con 0x7f6df809e590 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df809f6a0 con 0x7f6df809ea50 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6df8134ca0 con 0x7f6df813e7d0 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6e004d4640 10 monclient(hunting): authenticate will time out at 830.419728s 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6dfe270640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f6df809e590 0x7f6df81cd5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35276/0 (socket says 172.21.15.7:35276) 2024-05-08T15:47:02.315 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6dfe270640 1 -- 172.21.15.7:0/3663020917 learned_addr learned my addr 172.21.15.7:0/3663020917 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2820186986 0 0) 0x7f6df8134ca0 con 0x7f6df813e7d0 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): my global_id is 4198 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ddc003520 con 0x7f6df813e7d0 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1096437155 0 0) 0x7f6df809fbb0 con 0x7f6df809e590 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): my global_id is 74171 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df8134ca0 con 0x7f6df809e590 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099580027 0 0) 0x7f6df809f6a0 con 0x7f6df809ea50 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:02.316 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): my global_id is 74181 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df809fbb0 con 0x7f6df809ea50 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2940851774 0 0) 0x7f6ddc003520 con 0x7f6df813e7d0 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6df809f6a0 con 0x7f6df813e7d0 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6df0001c70 con 0x7f6df813e7d0 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:02.317 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 85298455 0 0) 0x7f6df8134ca0 con 0x7f6df809e590 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ddc003520 con 0x7f6df809e590 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3541649493 0 0) 0x7f6df809fbb0 con 0x7f6df809ea50 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6df8134ca0 con 0x7f6df809ea50 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6dec001a90 con 0x7f6df809e590 2024-05-08T15:47:02.318 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:02.319 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6df4004b20 con 0x7f6df809ea50 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:02.320 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4281846135 0 0) 0x7f6df809f6a0 con 0x7f6df813e7d0 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 >> v1:172.21.15.81:6790/0 conn(0x7f6df809ea50 legacy=0x7f6df81d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 >> v1:172.21.15.7:6790/0 conn(0x7f6df809e590 legacy=0x7f6df81cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:02.321 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 monclient: found mon.c 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df806fc00 con 0x7f6df813e7d0 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.305+0000 7f6deaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6e004d4640 5 monclient: authenticate success, global_id 4198 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6e004d4640 10 monclient: _renew_subs 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6e004d4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6df806fee0 con 0x7f6df813e7d0 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6df0002010 con 0x7f6df813e7d0 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6deaffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:02.322 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6df0005150 con 0x7f6df813e7d0 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6deaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6deaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6dfca6d640 4 set_mon_vals no callback set 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6deaffd640 10 monclient: dump: 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:02.323 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6e004d4640 10 monclient: _renew_subs 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6e004d4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f6df8135340 con 0x7f6df813e7d0 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.306+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 4) ==== 45472+0+0 (unknown 3410756102 0 0) 0x7f6df00104a0 con 0x7f6df813e7d0 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.307+0000 7f6e004d4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.307+0000 7f6e004d4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.307+0000 7f6e004d4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.307+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6df80a0930 con 0x7f6df813e7d0 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.307+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(12..12 src has 1..12) ==== 2627+0+0 (unknown 120488937 0 0) 0x7f6df0046b40 con 0x7f6df813e7d0 2024-05-08T15:47:02.324 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.307+0000 7f6dfe270640 10 monclient: get_auth_request con 0x7f6ddc03a430 auth_method 0 2024-05-08T15:47:02.325 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.311+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f6df80a0930 con 0x7f6df813e7d0 2024-05-08T15:47:02.325 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.311+0000 7f6deaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:02.325 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.311+0000 7f6deaffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:02.493 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.496+0000 7f6e004d4640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:02.493 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.496+0000 7f6e004d4640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:02.493 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.496+0000 7f6e004d4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:02.494 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.496+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6df804f5f0 con 0x7f6df813e7d0 2024-05-08T15:47:02.494 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.497+0000 7f6deaffd640 1 -- 172.21.15.7:0/3663020917 <== mon.4 v1:172.21.15.7:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+6874 (unknown 4250787923 0 3489806855) 0x7f6df804f5f0 con 0x7f6df813e7d0 2024-05-08T15:47:02.494 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.497+0000 7f6deaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:02.494 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.497+0000 7f6deaffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:02.494 INFO:teuthology.misc.health.smithi007.stdout: 2024-05-08T15:47:02.494 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":12,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:47:02.123093+0000","last_up_change":"2024-05-08T15:47:01.112469+0000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6800","nonce":601836183}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6801","nonce":601836183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6803","nonce":601836183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6802","nonce":601836183}]},"public_addr":"172.21.15.7:6800/601836183","cluster_addr":"172.21.15.7:6801/601836183","heartbeat_back_addr":"172.21.15.7:6803/601836183","heartbeat_front_addr":"172.21.15.7:6802/601836183","state":["exists","up"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6808","nonce":2598303896}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6809","nonce":2598303896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6811","nonce":2598303896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6810","nonce":2598303896}]},"public_addr":"172.21.15.7:6808/2598303896","cluster_addr":"172.21.15.7:6809/2598303896","heartbeat_back_addr":"172.21.15.7:6811/2598303896","heartbeat_front_addr":"172.21.15.7:6810/2598303896","state":["exists","up"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6802","nonce":1326537731}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6805","nonce":1326537731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6807","nonce":1326537731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6806","nonce":1326537731}]},"public_addr":"172.21.15.81:6802/1326537731","cluster_addr":"172.21.15.81:6805/1326537731","heartbeat_back_addr":"172.21.15.81:6807/1326537731","heartbeat_front_addr":"172.21.15.81:6806/1326537731","state":["exists","up"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6808","nonce":193838907}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6809","nonce":193838907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6811","nonce":193838907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6810","nonce":193838907}]},"public_addr":"172.21.15.81:6808/193838907","cluster_addr":"172.21.15.81:6809/193838907","heartbeat_back_addr":"172.21.15.81:6811/193838907","heartbeat_front_addr":"172.21.15.81:6810/193838907","state":["exists","up"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6800","nonce":385867939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6801","nonce":385867939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6804","nonce":385867939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6803","nonce":385867939}]},"public_addr":"172.21.15.81:6800/385867939","cluster_addr":"172.21.15.81:6801/385867939","heartbeat_back_addr":"172.21.15.81:6804/385867939","heartbeat_front_addr":"172.21.15.81:6803/385867939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.485746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.746567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.749309+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.745027+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.779789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 >> v1:172.21.15.81:6812/847627118 conn(0x7f6ddc03a430 legacy=0x7f6ddc03c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 10 monclient: shutdown 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 >> v1:172.21.15.7:6791/0 conn(0x7f6df813e7d0 legacy=0x7f6df81d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 shutdown_connections 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 >> 172.21.15.7:0/3663020917 conn(0x7f6df80a7020 msgr2=0x7f6df813dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 shutdown_connections 2024-05-08T15:47:02.496 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:02.499+0000 7f6e004d4640 1 -- 172.21.15.7:0/3663020917 wait complete. 2024-05-08T15:47:02.505 DEBUG:teuthology.misc:5 of 6 OSDs are up 2024-05-08T15:47:02.932 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-05-08T15:47:02.936+0000 7fcfff54c640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T15:47:02.938 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-05-08T15:47:02.942+0000 7fcff736c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T15:47:09.110 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T15:47:09.222 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.226+0000 7f2bd3c9a640 10 monclient: get_monmap_and_config 2024-05-08T15:47:09.222 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.226+0000 7f2bd3c9a640 10 monclient: build_initial_monmap 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.226+0000 7f2bd3c9a640 10 monclient: monmap: 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:epoch 0 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:47:09.227245+0000 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:47:09.227245+0000 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:09.223 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:09.224 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:09.224 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:09.224 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.231 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.226+0000 7f2bd3c9a640 1 Processor -- start 2024-05-08T15:47:09.231 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 1 -- start start 2024-05-08T15:47:09.231 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient: init 2024-05-08T15:47:09.231 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:09.231 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient: _add_conns ranks=[4,5,2,0,3,1,8,6,7] 2024-05-08T15:47:09.231 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient(hunting): picked mon.noname-c con 0x7f2bcc09ea50 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:09.231 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient(hunting): picked mon.noname-h con 0x7f2bcc09e590 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient(hunting): picked mon.noname-b con 0x7f2bcc13e7d0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc0a0090 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc0a0650 con 0x7f2bcc09ea50 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc0a0930 con 0x7f2bcc09e590 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd3c9a640 10 monclient(hunting): authenticate will time out at 837.342294s 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd2237640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f2bcc13e7d0 0x7f2bcc140bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44116/0 (socket says 172.21.15.7:44116) 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd1a36640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f2bcc09ea50 0x7f2bcc09ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:50986/0 (socket says 172.21.15.7:50986) 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.227+0000 7f2bd2237640 1 -- 172.21.15.7:0/2317609137 learned_addr learned my addr 172.21.15.7:0/2317609137 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 586206379 0 0) 0x7f2bcc0a0650 con 0x7f2bcc09ea50 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.232 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): my global_id is 4225 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bb8003520 con 0x7f2bcc09ea50 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3285945335 0 0) 0x7f2bcc0a0090 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient(hunting): my global_id is 74198 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bcc0a0650 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 269178785 0 0) 0x7f2bcc0a0650 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 >> v1:172.21.15.81:6791/0 conn(0x7f2bcc09e590 legacy=0x7f2bcc13d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 >> v1:172.21.15.7:6791/0 conn(0x7f2bcc09ea50 legacy=0x7f2bcc09ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: _finish_hunting 0 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 monclient: found mon.noname-b 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:09.233 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc0a61d0 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: _finish_auth 0 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: _check_auth_tickets 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bc4001ab0 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd3c9a640 5 monclient: authenticate success, global_id 74198 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd3c9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: dump: 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.234 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd3c9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2bc4001d10 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:09.235 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd3c9a640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 1 -- 172.21.15.7:0/2317609137 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bc4004f60 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd3c9a640 4 set_mon_vals no callback set 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.228+0000 7f2bd0a34640 10 monclient: dump: 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.236 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 10 monclient: shutdown 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/2317609137 >> v1:172.21.15.7:6790/0 conn(0x7f2bcc13e7d0 legacy=0x7f2bcc140bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/2317609137 shutdown_connections 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/2317609137 >> 172.21.15.7:0/2317609137 conn(0x7f2bcc0a7020 msgr2=0x7f2bcc0a7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/2317609137 shutdown_connections 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/2317609137 wait complete. 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 10 monclient: build_initial_monmap 2024-05-08T15:47:09.237 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.229+0000 7f2bd3c9a640 10 monclient: monmap: 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:epoch 0 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:created 0.000000 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:09.238 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 1 Processor -- start 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 1 -- start start 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient: init 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient: _add_conns ranks=[4,5,8,0,2,7,3,1,6] 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient(hunting): picked mon.noname-c con 0x7f2bcc09e590 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient(hunting): picked mon.noname-h con 0x7f2bcc09ea50 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient(hunting): picked mon.noname-e con 0x7f2bcc13e7d0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc06fe20 con 0x7f2bcc09e590 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc09f6a0 con 0x7f2bcc09ea50 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bcc140fa0 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:09.239 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd3c9a640 10 monclient(hunting): authenticate will time out at 837.345440s 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd1a36640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f2bcc09e590 0x7f2bcc13f810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:50992/0 (socket says 172.21.15.7:50992) 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.230+0000 7f2bd1a36640 1 -- 172.21.15.7:0/898124780 learned_addr learned my addr 172.21.15.7:0/898124780 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130370050 0 0) 0x7f2bcc06fe20 con 0x7f2bcc09e590 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): my global_id is 4234 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bb0003520 con 0x7f2bcc09e590 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292961953 0 0) 0x7f2bcc140fa0 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): my global_id is 4220 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bcc06fe20 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.240 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3106888893 0 0) 0x7f2bcc09f6a0 con 0x7f2bcc09ea50 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): my global_id is 4253 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bcc140fa0 con 0x7f2bcc09ea50 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1251993219 0 0) 0x7f2bb0003520 con 0x7f2bcc09e590 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2bcc09f6a0 con 0x7f2bcc09e590 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bc8001550 con 0x7f2bcc09e590 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): dump: 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.241 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:09.242 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3793140028 0 0) 0x7f2bcc06fe20 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2bb0003520 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bc40019b0 con 0x7f2bcc13e7d0 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): dump: 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.243 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4098626756 0 0) 0x7f2bcc140fa0 con 0x7f2bcc09ea50 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2bcc06fe20 con 0x7f2bcc09ea50 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bbc001a50 con 0x7f2bcc09ea50 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient(hunting): dump: 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:09.244 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1416318687 0 0) 0x7f2bcc09f6a0 con 0x7f2bcc09e590 2024-05-08T15:47:09.245 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 >> v1:172.21.15.7:6793/0 conn(0x7f2bcc13e7d0 legacy=0x7f2bcc1d53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 >> v1:172.21.15.81:6791/0 conn(0x7f2bcc09ea50 legacy=0x7f2bcc13ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 monclient: found mon.c 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bcc070100 con 0x7f2bcc09e590 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bd3c9a640 5 monclient: authenticate success, global_id 4234 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bd3c9a640 10 monclient: _renew_subs 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bd3c9a640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2bcc06f860 con 0x7f2bcc09e590 2024-05-08T15:47:09.246 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bd3c9a640 10 monclient: _renew_subs 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bd3c9a640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f2bcc072a70 con 0x7f2bcc09e590 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2bc8004a90 con 0x7f2bcc09e590 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bc8004f80 con 0x7f2bcc09e590 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc37fe640 4 set_mon_vals no callback set 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.231+0000 7f2bc27fc640 10 monclient: dump: 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:epoch 1 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.247 INFO:teuthology.misc.health.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr: 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.232+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2bc8010300 con 0x7f2bcc09e590 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.232+0000 7f2bd3c9a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.232+0000 7f2bd3c9a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.232+0000 7f2bd3c9a640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:09.248 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.232+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb4005080 con 0x7f2bcc09e590 2024-05-08T15:47:09.249 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.232+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(15..15 src has 1..15) ==== 3153+0+0 (unknown 2247761309 0 0) 0x7f2bc8046b90 con 0x7f2bcc09e590 2024-05-08T15:47:09.249 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.232+0000 7f2bd1235640 10 monclient: get_auth_request con 0x7f2bb003a4d0 auth_method 0 2024-05-08T15:47:09.249 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.237+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f2bc8019bb0 con 0x7f2bcc09e590 2024-05-08T15:47:09.249 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.237+0000 7f2bc27fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:09.249 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.237+0000 7f2bc27fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:09.419 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.423+0000 7f2bd3c9a640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:09.419 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.423+0000 7f2bd3c9a640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:09.419 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.423+0000 7f2bd3c9a640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:09.419 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.423+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2bb4005370 con 0x7f2bcc09e590 2024-05-08T15:47:09.420 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.423+0000 7f2bc27fc640 1 -- 172.21.15.7:0/898124780 <== mon.4 v1:172.21.15.7:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) ==== 74+0+8961 (unknown 4221056075 0 3709195823) 0x7f2bc801e470 con 0x7f2bcc09e590 2024-05-08T15:47:09.420 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.423+0000 7f2bc27fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:09.420 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.423+0000 7f2bc27fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:09.420 INFO:teuthology.misc.health.smithi007.stdout: 2024-05-08T15:47:09.420 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":15,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:47:05.943469+0000","last_up_change":"2024-05-08T15:47:03.939015+0000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T15:47:03.139226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6804","nonce":3231689356}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6805","nonce":3231689356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6807","nonce":3231689356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6806","nonce":3231689356}]},"public_addr":"172.21.15.7:6804/3231689356","cluster_addr":"172.21.15.7:6805/3231689356","heartbeat_back_addr":"172.21.15.7:6807/3231689356","heartbeat_front_addr":"172.21.15.7:6806/3231689356","state":["exists","up"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6800","nonce":601836183}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6801","nonce":601836183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6803","nonce":601836183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6802","nonce":601836183}]},"public_addr":"172.21.15.7:6800/601836183","cluster_addr":"172.21.15.7:6801/601836183","heartbeat_back_addr":"172.21.15.7:6803/601836183","heartbeat_front_addr":"172.21.15.7:6802/601836183","state":["exists","up"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6808","nonce":2598303896}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6809","nonce":2598303896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6811","nonce":2598303896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6810","nonce":2598303896}]},"public_addr":"172.21.15.7:6808/2598303896","cluster_addr":"172.21.15.7:6809/2598303896","heartbeat_back_addr":"172.21.15.7:6811/2598303896","heartbeat_front_addr":"172.21.15.7:6810/2598303896","state":["exists","up"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6802","nonce":1326537731}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6805","nonce":1326537731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6807","nonce":1326537731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6806","nonce":1326537731}]},"public_addr":"172.21.15.81:6802/1326537731","cluster_addr":"172.21.15.81:6805/1326537731","heartbeat_back_addr":"172.21.15.81:6807/1326537731","heartbeat_front_addr":"172.21.15.81:6806/1326537731","state":["exists","up"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6808","nonce":193838907}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6809","nonce":193838907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6811","nonce":193838907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6810","nonce":193838907}]},"public_addr":"172.21.15.81:6808/193838907","cluster_addr":"172.21.15.81:6809/193838907","heartbeat_back_addr":"172.21.15.81:6811/193838907","heartbeat_front_addr":"172.21.15.81:6810/193838907","state":["exists","up"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6800","nonce":385867939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6801","nonce":385867939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6804","nonce":385867939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6803","nonce":385867939}]},"public_addr":"172.21.15.81:6800/385867939","cluster_addr":"172.21.15.81:6801/385867939","heartbeat_back_addr":"172.21.15.81:6804/385867939","heartbeat_front_addr":"172.21.15.81:6803/385867939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.702217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.485746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.746567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.749309+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.745027+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.779789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:47:09.422 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 >> v1:172.21.15.81:6812/847627118 conn(0x7f2bb003a4d0 legacy=0x7f2bb003c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.422 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 10 monclient: shutdown 2024-05-08T15:47:09.422 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:09.422 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 >> v1:172.21.15.7:6791/0 conn(0x7f2bcc09e590 legacy=0x7f2bcc13f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.422 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 shutdown_connections 2024-05-08T15:47:09.422 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 >> 172.21.15.7:0/898124780 conn(0x7f2bcc0a7020 msgr2=0x7f2bcc131a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:09.423 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 shutdown_connections 2024-05-08T15:47:09.423 INFO:teuthology.misc.health.smithi007.stderr:2024-05-08T15:47:09.425+0000 7f2bd3c9a640 1 -- 172.21.15.7:0/898124780 wait complete. 2024-05-08T15:47:09.431 DEBUG:teuthology.misc:6 of 6 OSDs are up 2024-05-08T15:47:09.431 INFO:tasks.ceph:Creating RBD pool 2024-05-08T15:47:09.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-08T15:47:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.537+0000 7ff349fb6640 10 monclient: get_monmap_and_config 2024-05-08T15:47:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.537+0000 7ff349fb6640 10 monclient: build_initial_monmap 2024-05-08T15:47:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.537+0000 7ff349fb6640 10 monclient: monmap: 2024-05-08T15:47:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:09.534 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:09.538741+0000 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:09.538741+0000 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:09.536 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.538+0000 7ff349fb6640 1 Processor -- start 2024-05-08T15:47:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.538+0000 7ff349fb6640 1 -- start start 2024-05-08T15:47:09.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.538+0000 7ff349fb6640 10 monclient: init 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 10 monclient: _add_conns ranks=[3,1,4,7,0,2,5,8,6] 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 10 monclient(hunting): picked mon.noname-g con 0x7ff344137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 10 monclient(hunting): picked mon.noname-f con 0x7ff3440a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 10 monclient(hunting): picked mon.noname-c con 0x7ff3440a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3440a2c70 con 0x7ff3440a49a0 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3440a2f50 con 0x7ff344137940 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3440a1670 con 0x7ff3440a41b0 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 10 monclient(hunting): authenticate will time out at 837.653816s 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff343fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7ff3440a41b0 0x7ff344140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51000/0 (socket says 172.21.15.7:51000) 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff343fff640 1 -- 172.21.15.7:0/3926639680 learned_addr learned my addr 172.21.15.7:0/3926639680 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441850573 0 0) 0x7ff3440a1670 con 0x7ff3440a41b0 2024-05-08T15:47:09.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): my global_id is 4243 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff32c003520 con 0x7ff3440a41b0 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1202845341 0 0) 0x7ff3440a2c70 con 0x7ff3440a49a0 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): my global_id is 4213 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3440a1670 con 0x7ff3440a49a0 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407701605 0 0) 0x7ff3440a2f50 con 0x7ff344137940 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient(hunting): my global_id is 74199 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3440a2c70 con 0x7ff344137940 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 592671850 0 0) 0x7ff32c003520 con 0x7ff3440a41b0 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 >> v1:172.21.15.81:6790/0 conn(0x7ff344137940 legacy=0x7ff344137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 >> v1:172.21.15.81:6789/0 conn(0x7ff3440a49a0 legacy=0x7ff34413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 monclient: found mon.noname-c 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3440a39f0 con 0x7ff3440a41b0 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff33c001b30 con 0x7ff3440a41b0 2024-05-08T15:47:09.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 5 monclient: authenticate success, global_id 4243 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff3427fc640 10 monclient: dump: 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.547 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.539+0000 7ff349fb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff33c001d90 con 0x7ff3440a41b0 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff3427fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff3427fc640 1 -- 172.21.15.7:0/3926639680 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff33c004f60 con 0x7ff3440a41b0 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 4 set_mon_vals no callback set 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff3427fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff3427fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:09.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff3427fc640 10 monclient: dump: 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.549 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 10 monclient: shutdown 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 1 -- 172.21.15.7:0/3926639680 >> v1:172.21.15.7:6791/0 conn(0x7ff3440a41b0 legacy=0x7ff344140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 1 -- 172.21.15.7:0/3926639680 shutdown_connections 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 1 -- 172.21.15.7:0/3926639680 >> 172.21.15.7:0/3926639680 conn(0x7ff3440a9fa0 msgr2=0x7ff3440aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 1 -- 172.21.15.7:0/3926639680 shutdown_connections 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.540+0000 7ff349fb6640 1 -- 172.21.15.7:0/3926639680 wait complete. 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.541+0000 7ff349fb6640 10 monclient: build_initial_monmap 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.541+0000 7ff349fb6640 10 monclient: monmap: 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:09.550 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.541+0000 7ff349fb6640 1 Processor -- start 2024-05-08T15:47:09.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.541+0000 7ff349fb6640 1 -- start start 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.541+0000 7ff349fb6640 10 monclient: init 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 10 monclient: _add_conns ranks=[0,2,8,6,4,1,7,5,3] 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 10 monclient(hunting): picked mon.noname-a con 0x7ff3440a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 10 monclient(hunting): picked mon.noname-b con 0x7ff3440a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 10 monclient(hunting): picked mon.noname-e con 0x7ff344137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff344135880 con 0x7ff3440a41b0 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff344133890 con 0x7ff3440a49a0 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff344135b60 con 0x7ff344137940 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff3437fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7ff3440a41b0 0x7ff344135f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57266/0 (socket says 172.21.15.7:57266) 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff3437fe640 1 -- 172.21.15.7:0/2756050889 learned_addr learned my addr 172.21.15.7:0/2756050889 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff349fb6640 10 monclient(hunting): authenticate will time out at 837.656813s 2024-05-08T15:47:09.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496435512 0 0) 0x7ff344133890 con 0x7ff3440a49a0 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): my global_id is 74207 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff330003520 con 0x7ff3440a49a0 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034455902 0 0) 0x7ff344135b60 con 0x7ff344137940 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): my global_id is 4229 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff344133890 con 0x7ff344137940 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4028917659 0 0) 0x7ff344135880 con 0x7ff3440a41b0 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:09.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): my global_id is 4275 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff344135b60 con 0x7ff3440a41b0 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 654515175 0 0) 0x7ff344135b60 con 0x7ff3440a41b0 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff344135880 con 0x7ff3440a41b0 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3001730249 0 0) 0x7ff344133890 con 0x7ff344137940 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff344135b60 con 0x7ff344137940 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff334001970 con 0x7ff3440a41b0 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): dump: 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.554 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:09.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2986818513 0 0) 0x7ff330003520 con 0x7ff3440a49a0 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff344133890 con 0x7ff3440a49a0 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff33c001a30 con 0x7ff344137940 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): dump: 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.556 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff3380019d0 con 0x7ff3440a49a0 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.542+0000 7ff32bfff640 10 monclient(hunting): dump: 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.557 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 408170548 0 0) 0x7ff344135880 con 0x7ff3440a41b0 2024-05-08T15:47:09.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 >> v1:172.21.15.7:6793/0 conn(0x7ff344137940 legacy=0x7ff3441d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 >> v1:172.21.15.7:6790/0 conn(0x7ff3440a49a0 legacy=0x7ff3441d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 1 monclient: found mon.a 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3440a1f30 con 0x7ff3440a41b0 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: _finish_auth 0 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff349fb6640 5 monclient: authenticate success, global_id 4275 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff349fb6640 10 monclient: _renew_subs 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff349fb6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff3441355a0 con 0x7ff3440a41b0 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff349fb6640 10 monclient: _renew_subs 2024-05-08T15:47:09.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff349fb6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7ff3441d5a90 con 0x7ff3440a41b0 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff334001cd0 con 0x7ff3440a41b0 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff334003c00 con 0x7ff3440a41b0 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff341ffb640 4 set_mon_vals no callback set 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.543+0000 7ff32bfff640 10 monclient: dump: 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:09.560 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.544+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7ff334003e20 con 0x7ff3440a41b0 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.544+0000 7ff349fb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.544+0000 7ff349fb6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.544+0000 7ff349fb6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.544+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff324005080 con 0x7ff3440a41b0 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.544+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(15..15 src has 1..15) ==== 3153+0+0 (unknown 2247761309 0 0) 0x7ff334046ad0 con 0x7ff3440a41b0 2024-05-08T15:47:09.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.544+0000 7ff342ffd640 10 monclient: get_auth_request con 0x7ff33003a3e0 auth_method 0 2024-05-08T15:47:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.548+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7ff334015e20 con 0x7ff3440a41b0 2024-05-08T15:47:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.548+0000 7ff32bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:09.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.548+0000 7ff32bfff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:09.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.730+0000 7ff349fb6640 10 start_mon_command cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-05-08T15:47:09.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.730+0000 7ff349fb6640 10 monclient: _send_command 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-05-08T15:47:09.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.730+0000 7ff349fb6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:09.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:09.730+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7ff324005b80 con 0x7ff3440a41b0 2024-05-08T15:47:10.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.159+0000 7ff32bfff640 1 -- 172.21.15.7:0/2756050889 <== mon.0 v1:172.21.15.7:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) ==== 109+0+0 (unknown 231814880 0 0) 0x7ff334015720 con 0x7ff3440a41b0 2024-05-08T15:47:10.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.159+0000 7ff32bfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}] 2024-05-08T15:47:10.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.159+0000 7ff32bfff640 10 monclient: _finish_command 2 = system:0 pool 'rbd' created 2024-05-08T15:47:10.156 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'rbd' created 2024-05-08T15:47:10.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 >> v1:172.21.15.81:6812/847627118 conn(0x7ff33003a3e0 legacy=0x7ff33003c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:10.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 10 monclient: shutdown 2024-05-08T15:47:10.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:10.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 >> v1:172.21.15.7:6789/0 conn(0x7ff3440a41b0 legacy=0x7ff344135f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:10.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 shutdown_connections 2024-05-08T15:47:10.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 >> 172.21.15.7:0/2756050889 conn(0x7ff3440a9fa0 msgr2=0x7ff344130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:10.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 shutdown_connections 2024-05-08T15:47:10.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.166+0000 7ff349fb6640 1 -- 172.21.15.7:0/2756050889 wait complete. 2024-05-08T15:47:10.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.288+0000 7fca40a82640 10 monclient: get_monmap_and_config 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.288+0000 7fca40a82640 10 monclient: build_initial_monmap 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.288+0000 7fca40a82640 10 monclient: monmap: 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:10.289771+0000 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:10.289771+0000 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:10.286 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.289+0000 7fca40a82640 1 Processor -- start 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.289+0000 7fca40a82640 1 -- start start 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.289+0000 7fca40a82640 10 monclient: init 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 10 monclient: _add_conns ranks=[6,3,7,0,4,2,1,5,8] 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 10 monclient(hunting): picked mon.noname-d con 0x7fca3c0a5250 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 10 monclient(hunting): picked mon.noname-g con 0x7fca3c0a48c0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 10 monclient(hunting): picked mon.noname-i con 0x7fca3c13e340 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c0a28a0 con 0x7fca3c0a48c0 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c0a2b80 con 0x7fca3c0a5250 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c0a2e60 con 0x7fca3c13e340 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 10 monclient(hunting): authenticate will time out at 838.404801s 2024-05-08T15:47:10.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3a59c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fca3c0a5250 0x7fca3c0a5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60252/0 (socket says 172.21.15.7:60252) 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3a59c640 1 -- 172.21.15.7:0/1269432173 learned_addr learned my addr 172.21.15.7:0/1269432173 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1808763743 0 0) 0x7fca3c0a2b80 con 0x7fca3c0a5250 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): my global_id is 74229 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca24003520 con 0x7fca3c0a5250 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1042076371 0 0) 0x7fca3c0a28a0 con 0x7fca3c0a48c0 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): my global_id is 74208 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca3c0a2b80 con 0x7fca3c0a48c0 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551070870 0 0) 0x7fca3c0a2e60 con 0x7fca3c13e340 2024-05-08T15:47:10.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient(hunting): my global_id is 4228 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca3c0a28a0 con 0x7fca3c13e340 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 564381438 0 0) 0x7fca24003520 con 0x7fca3c0a5250 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 >> v1:172.21.15.81:6792/0 conn(0x7fca3c13e340 legacy=0x7fca3c140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 >> v1:172.21.15.81:6790/0 conn(0x7fca3c0a48c0 legacy=0x7fca3c13cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 monclient: found mon.noname-d 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c0a42c0 con 0x7fca3c0a5250 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient: _finish_auth 0 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:10.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 5 monclient: authenticate success, global_id 74229 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca2c0017b0 con 0x7fca3c0a5250 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca3959a640 10 monclient: dump: 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:10.298 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.290+0000 7fca40a82640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fca2c001a60 con 0x7fca3c0a5250 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca3959a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 4 set_mon_vals no callback set 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca3959a640 1 -- 172.21.15.7:0/1269432173 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca2c004be0 con 0x7fca3c0a5250 2024-05-08T15:47:10.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca3959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca3959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca3959a640 10 monclient: dump: 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:10.300 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 10 monclient: shutdown 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 1 -- 172.21.15.7:0/1269432173 >> v1:172.21.15.7:6792/0 conn(0x7fca3c0a5250 legacy=0x7fca3c0a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 1 -- 172.21.15.7:0/1269432173 shutdown_connections 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 1 -- 172.21.15.7:0/1269432173 >> 172.21.15.7:0/1269432173 conn(0x7fca3c0a9ec0 msgr2=0x7fca3c0aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 1 -- 172.21.15.7:0/1269432173 shutdown_connections 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.291+0000 7fca40a82640 1 -- 172.21.15.7:0/1269432173 wait complete. 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.292+0000 7fca40a82640 10 monclient: build_initial_monmap 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.292+0000 7fca40a82640 10 monclient: monmap: 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:10.301 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.292+0000 7fca40a82640 1 Processor -- start 2024-05-08T15:47:10.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.292+0000 7fca40a82640 1 -- start start 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.292+0000 7fca40a82640 10 monclient: init 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 10 monclient: _add_conns ranks=[5,3,7,2,4,1,8,0,6] 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 10 monclient(hunting): picked mon.noname-h con 0x7fca3c0a48c0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 10 monclient(hunting): picked mon.noname-g con 0x7fca3c0a5250 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 10 monclient(hunting): picked mon.noname-i con 0x7fca3c13e340 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c13a250 con 0x7fca3c0a5250 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c1cd350 con 0x7fca3c0a48c0 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca3c13d420 con 0x7fca3c13e340 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca40a82640 10 monclient(hunting): authenticate will time out at 838.407829s 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca39d9b640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fca3c0a5250 0x7fca3c1ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50560/0 (socket says 172.21.15.7:50560) 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca39d9b640 1 -- 172.21.15.7:0/1182778659 learned_addr learned my addr 172.21.15.7:0/1182778659 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:10.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087728899 0 0) 0x7fca3c13a250 con 0x7fca3c0a5250 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): my global_id is 74217 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca1c003520 con 0x7fca3c0a5250 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2682119768 0 0) 0x7fca3c13d420 con 0x7fca3c13e340 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): my global_id is 4237 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca3c13a250 con 0x7fca3c13e340 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733942598 0 0) 0x7fca3c1cd350 con 0x7fca3c0a48c0 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:10.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): my global_id is 4262 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca3c13d420 con 0x7fca3c0a48c0 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3234262587 0 0) 0x7fca3c13a250 con 0x7fca3c13e340 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca3c1cd350 con 0x7fca3c13e340 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca34001d30 con 0x7fca3c13e340 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.293+0000 7fca2affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): dump: 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:10.305 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 79100486 0 0) 0x7fca1c003520 con 0x7fca3c0a5250 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca3c13a250 con 0x7fca3c0a5250 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 944919659 0 0) 0x7fca3c13d420 con 0x7fca3c0a48c0 2024-05-08T15:47:10.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca1c003520 con 0x7fca3c0a48c0 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca300019d0 con 0x7fca3c0a5250 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): dump: 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:10.307 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca2c004b20 con 0x7fca3c0a48c0 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient(hunting): dump: 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:10.308 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1773621123 0 0) 0x7fca3c1cd350 con 0x7fca3c13e340 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 >> v1:172.21.15.81:6791/0 conn(0x7fca3c0a48c0 legacy=0x7fca3c1cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 >> v1:172.21.15.81:6790/0 conn(0x7fca3c0a5250 legacy=0x7fca3c1ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:10.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 monclient: found mon.i 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c0a1ee0 con 0x7fca3c13e340 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: _finish_auth 0 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca40a82640 5 monclient: authenticate success, global_id 4237 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca40a82640 10 monclient: _renew_subs 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca40a82640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c0a1b20 con 0x7fca3c13e340 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca40a82640 10 monclient: _renew_subs 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca40a82640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fca3c13cc60 con 0x7fca3c13e340 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fca34002080 con 0x7fca3c13e340 2024-05-08T15:47:10.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca34004000 con 0x7fca3c13e340 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca38d99640 4 set_mon_vals no callback set 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.294+0000 7fca2affd640 10 monclient: dump: 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:10.311 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.295+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fca3400f400 con 0x7fca3c13e340 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.295+0000 7fca40a82640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.295+0000 7fca40a82640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.295+0000 7fca40a82640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.295+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca20005080 con 0x7fca3c13e340 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.295+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(16..16 src has 1..16) ==== 3495+0+0 (unknown 2242147014 0 0) 0x7fca34045ed0 con 0x7fca3c13e340 2024-05-08T15:47:10.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.295+0000 7fca3a59c640 10 monclient: get_auth_request con 0x7fca1c03a3d0 auth_method 0 2024-05-08T15:47:10.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.300+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fca340150e0 con 0x7fca3c13e340 2024-05-08T15:47:10.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.300+0000 7fca2affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:10.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.300+0000 7fca2affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:10.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.484+0000 7fca40a82640 10 start_mon_command cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-05-08T15:47:10.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.484+0000 7fca40a82640 10 monclient: _send_command 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-05-08T15:47:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.484+0000 7fca40a82640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:10.484+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7fca20005370 con 0x7fca3c13e340 2024-05-08T15:47:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.170+0000 7fca2affd640 1 -- 172.21.15.7:0/1182778659 <== mon.7 v1:172.21.15.81:6792/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) ==== 173+0+0 (unknown 2255673937 0 0) 0x7fca340149e0 con 0x7fca3c13e340 2024-05-08T15:47:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.171+0000 7fca2affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}] 2024-05-08T15:47:11.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.171+0000 7fca2affd640 10 monclient: _finish_command 2 = system:0 enabled application 'rbd' on pool 'rbd' 2024-05-08T15:47:11.168 INFO:teuthology.orchestra.run.smithi007.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-08T15:47:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 >> v1:172.21.15.81:6812/847627118 conn(0x7fca1c03a3d0 legacy=0x7fca1c03c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 10 monclient: shutdown 2024-05-08T15:47:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:11.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 >> v1:172.21.15.81:6792/0 conn(0x7fca3c13e340 legacy=0x7fca3c1d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 shutdown_connections 2024-05-08T15:47:11.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 >> 172.21.15.7:0/1182778659 conn(0x7fca3c0a9ec0 msgr2=0x7fca3c131c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:11.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 shutdown_connections 2024-05-08T15:47:11.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.175+0000 7fca40a82640 1 -- 172.21.15.7:0/1182778659 wait complete. 2024-05-08T15:47:11.185 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T15:47:11.185 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T15:47:11.185 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T15:47:11.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T15:47:11.186 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T15:47:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.298+0000 7f6ab7d80640 10 monclient: get_monmap_and_config 2024-05-08T15:47:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.298+0000 7f6ab7d80640 10 monclient: build_initial_monmap 2024-05-08T15:47:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.299+0000 7f6ab7d80640 10 monclient: monmap: 2024-05-08T15:47:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.295 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:11.299980+0000 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:11.299980+0000 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:11.296 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:11.297 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.299+0000 7f6ab7d80640 1 Processor -- start 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.299+0000 7f6ab7d80640 1 -- start start 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.299+0000 7f6ab7d80640 10 monclient: init 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 10 monclient: _add_conns ranks=[1,7,6,8,3,0,4,5,2] 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 10 monclient(hunting): picked mon.noname-f con 0x7f6ab0137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 10 monclient(hunting): picked mon.noname-i con 0x7f6ab00a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 10 monclient(hunting): picked mon.noname-d con 0x7f6ab00a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab00a2c70 con 0x7f6ab0137940 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab00a2f50 con 0x7f6ab00a41b0 2024-05-08T15:47:11.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab00a1670 con 0x7f6ab00a49a0 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab7d80640 10 monclient(hunting): authenticate will time out at 839.414971s 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab631d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f6ab00a41b0 0x7f6ab0140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60262/0 (socket says 172.21.15.7:60262) 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab631d640 1 -- 172.21.15.7:0/2572347643 learned_addr learned my addr 172.21.15.7:0/2572347643 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477906687 0 0) 0x7f6ab00a2f50 con 0x7f6ab00a41b0 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): my global_id is 74238 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a9c003520 con 0x7f6ab00a41b0 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2502658276 0 0) 0x7f6ab00a2c70 con 0x7f6ab0137940 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): my global_id is 4222 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab00a2f50 con 0x7f6ab0137940 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3442373872 0 0) 0x7f6ab00a1670 con 0x7f6ab00a49a0 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 10 monclient(hunting): my global_id is 4246 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab00a2c70 con 0x7f6ab00a49a0 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.300+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 7209391 0 0) 0x7f6a9c003520 con 0x7f6ab00a41b0 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 >> v1:172.21.15.81:6792/0 conn(0x7f6ab00a49a0 legacy=0x7f6ab013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 >> v1:172.21.15.81:6789/0 conn(0x7f6ab0137940 legacy=0x7f6ab0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 1 monclient: found mon.noname-d 2024-05-08T15:47:11.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab00a39f0 con 0x7f6ab00a41b0 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: _finish_auth 0 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6aa8001b30 con 0x7f6ab00a41b0 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 5 monclient: authenticate success, global_id 74238 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: dump: 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.310 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6aa8001d90 con 0x7f6ab00a41b0 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 1 -- 172.21.15.7:0/2572347643 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6aa8004f60 con 0x7f6ab00a41b0 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 4 set_mon_vals no callback set 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab4b1a640 10 monclient: dump: 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.312 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 10 monclient: shutdown 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2572347643 >> v1:172.21.15.7:6792/0 conn(0x7f6ab00a41b0 legacy=0x7f6ab0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2572347643 shutdown_connections 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2572347643 >> 172.21.15.7:0/2572347643 conn(0x7f6ab00a9fa0 msgr2=0x7f6ab00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2572347643 shutdown_connections 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.301+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2572347643 wait complete. 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.302+0000 7f6ab7d80640 10 monclient: build_initial_monmap 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.302+0000 7f6ab7d80640 10 monclient: monmap: 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:11.314 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.302+0000 7f6ab7d80640 1 Processor -- start 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.302+0000 7f6ab7d80640 1 -- start start 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.302+0000 7f6ab7d80640 10 monclient: init 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 10 monclient: _add_conns ranks=[5,2,6,7,0,8,1,3,4] 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 10 monclient(hunting): picked mon.noname-h con 0x7f6ab00a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 10 monclient(hunting): picked mon.noname-b con 0x7f6ab00a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:11.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 10 monclient(hunting): picked mon.noname-d con 0x7f6ab0137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab01376b0 con 0x7f6ab00a49a0 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab00a0f60 con 0x7f6ab00a41b0 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ab0135870 con 0x7f6ab0137940 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab7d80640 10 monclient(hunting): authenticate will time out at 839.418014s 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab631d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f6ab0137940 0x7f6ab01d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60276/0 (socket says 172.21.15.7:60276) 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab531b640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f6ab00a49a0 0x7f6ab0132920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44122/0 (socket says 172.21.15.7:44122) 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6ab631d640 1 -- 172.21.15.7:0/2689872475 learned_addr learned my addr 172.21.15.7:0/2689872475 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1743950523 0 0) 0x7f6ab0135870 con 0x7f6ab0137940 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): my global_id is 74247 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a94003520 con 0x7f6ab0137940 2024-05-08T15:47:11.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1453099411 0 0) 0x7f6ab01376b0 con 0x7f6ab00a49a0 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): my global_id is 74216 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab0135870 con 0x7f6ab00a49a0 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 463032804 0 0) 0x7f6ab00a0f60 con 0x7f6ab00a41b0 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 10 monclient(hunting): my global_id is 4271 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ab01376b0 con 0x7f6ab00a41b0 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4064774202 0 0) 0x7f6ab0135870 con 0x7f6ab00a49a0 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.303+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ab00a0f60 con 0x7f6ab00a49a0 2024-05-08T15:47:11.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3949324095 0 0) 0x7f6a94003520 con 0x7f6ab0137940 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6ab0135870 con 0x7f6ab0137940 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6aa0001a90 con 0x7f6ab00a49a0 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): dump: 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.318 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6aa8004ea0 con 0x7f6ab0137940 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): dump: 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.319 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1150682206 0 0) 0x7f6ab01376b0 con 0x7f6ab00a41b0 2024-05-08T15:47:11.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a94003520 con 0x7f6ab00a41b0 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6aac001970 con 0x7f6ab00a41b0 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient(hunting): dump: 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.321 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2584714221 0 0) 0x7f6ab00a0f60 con 0x7f6ab00a49a0 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 >> v1:172.21.15.7:6792/0 conn(0x7f6ab0137940 legacy=0x7f6ab01d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 >> v1:172.21.15.81:6791/0 conn(0x7f6ab00a41b0 legacy=0x7f6ab0132210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 monclient: found mon.b 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:11.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab0133030 con 0x7f6ab00a49a0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6ab7d80640 5 monclient: authenticate success, global_id 74216 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6ab7d80640 10 monclient: _renew_subs 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6ab7d80640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab0135590 con 0x7f6ab00a49a0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6ab7d80640 10 monclient: _renew_subs 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6ab7d80640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f6ab01365f0 con 0x7f6ab00a49a0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6aa00038c0 con 0x7f6ab00a49a0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6aa0003db0 con 0x7f6ab00a49a0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa7fff640 4 set_mon_vals no callback set 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.304+0000 7f6aa67fc640 10 monclient: dump: 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.324 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.305+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6aa0003fd0 con 0x7f6ab00a49a0 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.305+0000 7f6ab7d80640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.305+0000 7f6ab7d80640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.305+0000 7f6ab7d80640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.305+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a98005080 con 0x7f6ab00a49a0 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.305+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(17..17 src has 1..17) ==== 3506+0+0 (unknown 993385953 0 0) 0x7f6aa0046c80 con 0x7f6ab00a49a0 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.305+0000 7f6ab5b1c640 10 monclient: get_auth_request con 0x7f6a9403a3b0 auth_method 0 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.310+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f6aa0015e90 con 0x7f6ab00a49a0 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.310+0000 7f6aa67fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:11.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.310+0000 7f6aa67fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.541+0000 7f6ab7d80640 10 start_mon_command cmd=[{"prefix": "mgr dump", "format": "json"}] 2024-05-08T15:47:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.541+0000 7f6ab7d80640 10 monclient: _send_command 2 [{"prefix": "mgr dump", "format": "json"}] 2024-05-08T15:47:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.541+0000 7f6ab7d80640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:11.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.541+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f6a98005370 con 0x7f6ab00a49a0 2024-05-08T15:47:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.542+0000 7f6aa67fc640 1 -- 172.21.15.7:0/2689872475 <== mon.2 v1:172.21.15.7:6790/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+87135 (unknown 481731318 0 1322020090) 0x7f6aa0019b40 con 0x7f6ab00a49a0 2024-05-08T15:47:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.542+0000 7f6aa67fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mgr dump", "format": "json"}] 2024-05-08T15:47:11.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.542+0000 7f6aa67fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:11.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:11.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.544+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 >> v1:172.21.15.81:6812/847627118 conn(0x7f6a9403a3b0 legacy=0x7f6a9403c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.544+0000 7f6ab7d80640 10 monclient: shutdown 2024-05-08T15:47:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.544+0000 7f6ab7d80640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.544+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 >> v1:172.21.15.7:6790/0 conn(0x7f6ab00a49a0 legacy=0x7f6ab0132920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.545+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 shutdown_connections 2024-05-08T15:47:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.545+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 >> 172.21.15.7:0/2689872475 conn(0x7f6ab00a9fa0 msgr2=0x7f6ab00a8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.545+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 shutdown_connections 2024-05-08T15:47:11.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.545+0000 7f6ab7d80640 1 -- 172.21.15.7:0/2689872475 wait complete. 2024-05-08T15:47:11.552 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6812","nonce":847627118}]},"active_addr":"172.21.15.81:6812/847627118","active_change":"2024-05-08T15:46:57.089537+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":769386419}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":880525561}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":2018048181}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":792180259}]}]} 2024-05-08T15:47:11.557 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T15:47:11.557 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T15:47:11.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T15:47:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.664+0000 7f31d6a7b640 10 monclient: get_monmap_and_config 2024-05-08T15:47:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.664+0000 7f31d6a7b640 10 monclient: build_initial_monmap 2024-05-08T15:47:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.664+0000 7f31d6a7b640 10 monclient: monmap: 2024-05-08T15:47:11.660 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:11.665206+0000 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:11.665206+0000 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:11.661 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:11.662 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.664+0000 7f31d6a7b640 1 Processor -- start 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 1 -- start start 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient: init 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient: _add_conns ranks=[1,6,8,2,5,7,3,0,4] 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient(hunting): picked mon.noname-f con 0x7f31d00a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient(hunting): picked mon.noname-d con 0x7f31d00a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient(hunting): picked mon.noname-e con 0x7f31d013e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d00a1660 con 0x7f31d00a53a0 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d00a28f0 con 0x7f31d00a5780 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d00a2bd0 con 0x7f31d013e310 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d6a7b640 10 monclient(hunting): authenticate will time out at 839.780127s 2024-05-08T15:47:11.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d5018640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f31d013e310 0x7f31d0140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54476/0 (socket says 172.21.15.7:54476) 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cffff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f31d00a5780 0x7f31d013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60284/0 (socket says 172.21.15.7:60284) 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31d5018640 1 -- 172.21.15.7:0/3363173593 learned_addr learned my addr 172.21.15.7:0/3363173593 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1604230547 0 0) 0x7f31d00a28f0 con 0x7f31d00a5780 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): my global_id is 74256 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31bc003520 con 0x7f31d00a5780 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446520690 0 0) 0x7f31d00a2bd0 con 0x7f31d013e310 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 10 monclient(hunting): my global_id is 4238 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d00a28f0 con 0x7f31d013e310 2024-05-08T15:47:11.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.665+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253702303 0 0) 0x7f31d00a1660 con 0x7f31d00a53a0 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient(hunting): my global_id is 4231 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d00a2bd0 con 0x7f31d00a53a0 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3513204209 0 0) 0x7f31bc003520 con 0x7f31d00a5780 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 >> v1:172.21.15.7:6793/0 conn(0x7f31d013e310 legacy=0x7f31d0140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 >> v1:172.21.15.81:6789/0 conn(0x7f31d00a53a0 legacy=0x7f31d00a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 monclient: found mon.noname-d 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d00a6730 con 0x7f31d00a5780 2024-05-08T15:47:11.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31b80016e0 con 0x7f31d00a5780 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 5 monclient: authenticate success, global_id 74256 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: dump: 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.675 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f31b8001940 con 0x7f31d00a5780 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 1 -- 172.21.15.7:0/3363173593 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31b8004b20 con 0x7f31d00a5780 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:11.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 4 set_mon_vals no callback set 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31cf7fe640 10 monclient: dump: 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.677 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 10 monclient: shutdown 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3363173593 >> v1:172.21.15.7:6792/0 conn(0x7f31d00a5780 legacy=0x7f31d013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3363173593 shutdown_connections 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3363173593 >> 172.21.15.7:0/3363173593 conn(0x7f31d00a9fa0 msgr2=0x7f31d00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3363173593 shutdown_connections 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.666+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3363173593 wait complete. 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.667+0000 7f31d6a7b640 10 monclient: build_initial_monmap 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.667+0000 7f31d6a7b640 10 monclient: monmap: 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:11.678 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.667+0000 7f31d6a7b640 1 Processor -- start 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.667+0000 7f31d6a7b640 1 -- start start 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.667+0000 7f31d6a7b640 10 monclient: init 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 10 monclient: _add_conns ranks=[8,1,7,4,0,3,5,6,2] 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 10 monclient(hunting): picked mon.noname-e con 0x7f31d00a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 10 monclient(hunting): picked mon.noname-f con 0x7f31d00a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 10 monclient(hunting): picked mon.noname-i con 0x7f31d013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d01c9d70 con 0x7f31d00a5780 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d00a43c0 con 0x7f31d013e310 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31d0135890 con 0x7f31d00a53a0 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d6a7b640 10 monclient(hunting): authenticate will time out at 839.783232s 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d4817640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f31d00a53a0 0x7f31d01c8f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54480/0 (socket says 172.21.15.7:54480) 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31d4817640 1 -- 172.21.15.7:0/3871828446 learned_addr learned my addr 172.21.15.7:0/3871828446 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:11.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715748058 0 0) 0x7f31d0135890 con 0x7f31d00a53a0 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31cd7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31cd7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31cd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31cd7fa640 10 monclient(hunting): my global_id is 4247 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.668+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31b0003520 con 0x7f31d00a53a0 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 70389099 0 0) 0x7f31d01c9d70 con 0x7f31d00a5780 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): my global_id is 4240 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d0135890 con 0x7f31d00a5780 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768700339 0 0) 0x7f31d00a43c0 con 0x7f31d013e310 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): my global_id is 4255 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31d01c9d70 con 0x7f31d013e310 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2482597989 0 0) 0x7f31b0003520 con 0x7f31d00a53a0 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f31d00a43c0 con 0x7f31d00a53a0 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31c0001550 con 0x7f31d00a53a0 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): dump: 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.682 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:11.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2819540768 0 0) 0x7f31d0135890 con 0x7f31d00a5780 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f31b0003520 con 0x7f31d00a5780 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31b80012a0 con 0x7f31d00a5780 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient(hunting): dump: 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.684 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 685337623 0 0) 0x7f31d00a43c0 con 0x7f31d00a53a0 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 >> v1:172.21.15.81:6792/0 conn(0x7f31d013e310 legacy=0x7f31d01d4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 >> v1:172.21.15.81:6789/0 conn(0x7f31d00a5780 legacy=0x7f31d01c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 monclient: found mon.e 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:11.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d0137b80 con 0x7f31d00a53a0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: _finish_auth 0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31d6a7b640 5 monclient: authenticate success, global_id 4247 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31d6a7b640 10 monclient: _renew_subs 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31d6a7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f31d0136580 con 0x7f31d00a53a0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31d6a7b640 10 monclient: _renew_subs 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31d6a7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f31d0138520 con 0x7f31d00a53a0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f31c00038a0 con 0x7f31d00a53a0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31c0003d90 con 0x7f31d00a53a0 2024-05-08T15:47:11.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31ceffd640 4 set_mon_vals no callback set 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.669+0000 7f31cd7fa640 10 monclient: dump: 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.687 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.670+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f31c0003f30 con 0x7f31d00a53a0 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.670+0000 7f31d6a7b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.670+0000 7f31d6a7b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.670+0000 7f31d6a7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.670+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b4005080 con 0x7f31d00a53a0 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.670+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(17..17 src has 1..17) ==== 3506+0+0 (unknown 993385953 0 0) 0x7f31c0046d90 con 0x7f31d00a53a0 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.670+0000 7f31cffff640 10 monclient: get_auth_request con 0x7f31b003a220 auth_method 0 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.675+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f31c0015ea0 con 0x7f31d00a53a0 2024-05-08T15:47:11.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.675+0000 7f31cd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:11.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.675+0000 7f31cd7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.863+0000 7f31d6a7b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.863+0000 7f31d6a7b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.863+0000 7f31d6a7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:11.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.863+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31b4005370 con 0x7f31d00a53a0 2024-05-08T15:47:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.863+0000 7f31cd7fa640 1 -- 172.21.15.7:0/3871828446 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+10748 (unknown 3225238715 0 1574599536) 0x7f31c0019b50 con 0x7f31d00a53a0 2024-05-08T15:47:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.863+0000 7f31cd7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.863+0000 7f31cd7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:11.860 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:11.860 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:47:11.161127+0000","last_up_change":"2024-05-08T15:47:03.939015+0000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T15:47:03.139226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T15:47:09.761977+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6804","nonce":3231689356}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6805","nonce":3231689356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6807","nonce":3231689356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6806","nonce":3231689356}]},"public_addr":"172.21.15.7:6804/3231689356","cluster_addr":"172.21.15.7:6805/3231689356","heartbeat_back_addr":"172.21.15.7:6807/3231689356","heartbeat_front_addr":"172.21.15.7:6806/3231689356","state":["exists","up"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6800","nonce":601836183}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6801","nonce":601836183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6803","nonce":601836183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6802","nonce":601836183}]},"public_addr":"172.21.15.7:6800/601836183","cluster_addr":"172.21.15.7:6801/601836183","heartbeat_back_addr":"172.21.15.7:6803/601836183","heartbeat_front_addr":"172.21.15.7:6802/601836183","state":["exists","up"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6808","nonce":2598303896}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6809","nonce":2598303896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6811","nonce":2598303896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6810","nonce":2598303896}]},"public_addr":"172.21.15.7:6808/2598303896","cluster_addr":"172.21.15.7:6809/2598303896","heartbeat_back_addr":"172.21.15.7:6811/2598303896","heartbeat_front_addr":"172.21.15.7:6810/2598303896","state":["exists","up"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6802","nonce":1326537731}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6805","nonce":1326537731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6807","nonce":1326537731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6806","nonce":1326537731}]},"public_addr":"172.21.15.81:6802/1326537731","cluster_addr":"172.21.15.81:6805/1326537731","heartbeat_back_addr":"172.21.15.81:6807/1326537731","heartbeat_front_addr":"172.21.15.81:6806/1326537731","state":["exists","up"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6808","nonce":193838907}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6809","nonce":193838907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6811","nonce":193838907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6810","nonce":193838907}]},"public_addr":"172.21.15.81:6808/193838907","cluster_addr":"172.21.15.81:6809/193838907","heartbeat_back_addr":"172.21.15.81:6811/193838907","heartbeat_front_addr":"172.21.15.81:6810/193838907","state":["exists","up"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6800","nonce":385867939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6801","nonce":385867939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6804","nonce":385867939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6803","nonce":385867939}]},"public_addr":"172.21.15.81:6800/385867939","cluster_addr":"172.21.15.81:6801/385867939","heartbeat_back_addr":"172.21.15.81:6804/385867939","heartbeat_front_addr":"172.21.15.81:6803/385867939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.702217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.485746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.746567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.749309+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.745027+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.779789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:47:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.865+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 >> v1:172.21.15.81:6812/847627118 conn(0x7f31b003a220 legacy=0x7f31b003c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.865+0000 7f31d6a7b640 10 monclient: shutdown 2024-05-08T15:47:11.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.865+0000 7f31d6a7b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.865+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 >> v1:172.21.15.7:6793/0 conn(0x7f31d00a53a0 legacy=0x7f31d01c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.865+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 shutdown_connections 2024-05-08T15:47:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.865+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 >> 172.21.15.7:0/3871828446 conn(0x7f31d00a9fa0 msgr2=0x7f31d00a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.865+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 shutdown_connections 2024-05-08T15:47:11.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.866+0000 7f31d6a7b640 1 -- 172.21.15.7:0/3871828446 wait complete. 2024-05-08T15:47:11.873 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T15:47:11.873 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T15:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.981+0000 7fd5e194b640 10 monclient: get_monmap_and_config 2024-05-08T15:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.981+0000 7fd5e194b640 10 monclient: build_initial_monmap 2024-05-08T15:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.981+0000 7fd5e194b640 10 monclient: monmap: 2024-05-08T15:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.977 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:11.982096+0000 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:11.982096+0000 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:11.978 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.981+0000 7fd5e194b640 1 Processor -- start 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.981+0000 7fd5e194b640 1 -- start start 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.981+0000 7fd5e194b640 10 monclient: init 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 10 monclient: _add_conns ranks=[2,4,0,1,3,5,7,6,8] 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 10 monclient(hunting): picked mon.noname-b con 0x7fd5dc0a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 10 monclient(hunting): picked mon.noname-c con 0x7fd5dc0a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 10 monclient(hunting): picked mon.noname-a con 0x7fd5dc13e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5dc0a1660 con 0x7fd5dc13e310 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5dc0a28f0 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5dc0a2bd0 con 0x7fd5dc0a5780 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:11.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5e194b640 10 monclient(hunting): authenticate will time out at 840.097096s 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5daffd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd5dc0a53a0 0x7fd5dc0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44138/0 (socket says 172.21.15.7:44138) 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5daffd640 1 -- 172.21.15.7:0/1937918334 learned_addr learned my addr 172.21.15.7:0/1937918334 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5da7fc640 1 --1- 172.21.15.7:0/1937918334 >> v1:172.21.15.7:6791/0 conn(0x7fd5dc0a5780 0x7fd5dc13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51014/0 (socket says 172.21.15.7:51014) 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 370212841 0 0) 0x7fd5dc0a1660 con 0x7fd5dc13e310 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): my global_id is 4284 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5c4003520 con 0x7fd5dc13e310 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655781180 0 0) 0x7fd5dc0a2bd0 con 0x7fd5dc0a5780 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): my global_id is 4252 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5dc0a1660 con 0x7fd5dc0a5780 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3102166916 0 0) 0x7fd5dc0a28f0 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 10 monclient(hunting): my global_id is 74225 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.982+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5dc0a2bd0 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4188519126 0 0) 0x7fd5c4003520 con 0x7fd5dc13e310 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 >> v1:172.21.15.7:6791/0 conn(0x7fd5dc0a5780 legacy=0x7fd5dc13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 >> v1:172.21.15.7:6790/0 conn(0x7fd5dc0a53a0 legacy=0x7fd5dc0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 monclient: found mon.noname-a 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:11.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc0a6730 con 0x7fd5dc13e310 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 5 monclient: authenticate success, global_id 4284 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd5d4001a60 con 0x7fd5dc13e310 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: dump: 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.989 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.990 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd5d4001cc0 con 0x7fd5dc13e310 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 1 -- 172.21.15.7:0/1937918334 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd5d4001f30 con 0x7fd5dc13e310 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5d9ffb640 10 monclient: dump: 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.991 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 4 set_mon_vals no callback set 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 10 monclient: shutdown 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 1 -- 172.21.15.7:0/1937918334 >> v1:172.21.15.7:6789/0 conn(0x7fd5dc13e310 legacy=0x7fd5dc140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 1 -- 172.21.15.7:0/1937918334 shutdown_connections 2024-05-08T15:47:11.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 1 -- 172.21.15.7:0/1937918334 >> 172.21.15.7:0/1937918334 conn(0x7fd5dc0a9fa0 msgr2=0x7fd5dc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 1 -- 172.21.15.7:0/1937918334 shutdown_connections 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.983+0000 7fd5e194b640 1 -- 172.21.15.7:0/1937918334 wait complete. 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.984+0000 7fd5e194b640 10 monclient: build_initial_monmap 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.984+0000 7fd5e194b640 10 monclient: monmap: 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:11.993 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.984+0000 7fd5e194b640 1 Processor -- start 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.984+0000 7fd5e194b640 1 -- start start 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.984+0000 7fd5e194b640 10 monclient: init 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 10 monclient: _add_conns ranks=[4,0,6,3,5,8,2,7,1] 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 10 monclient(hunting): picked mon.noname-c con 0x7fd5dc0a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 10 monclient(hunting): picked mon.noname-a con 0x7fd5dc0a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 10 monclient(hunting): picked mon.noname-d con 0x7fd5dc13e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:11.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5dc1ccea0 con 0x7fd5dc0a5780 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5dc0a1e00 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5dc0a43c0 con 0x7fd5dc13e310 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5e194b640 10 monclient(hunting): authenticate will time out at 840.100003s 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5daffd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fd5dc0a53a0 0x7fd5dc1cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51018/0 (socket says 172.21.15.7:51018) 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5daffd640 1 -- 172.21.15.7:0/1274915505 learned_addr learned my addr 172.21.15.7:0/1274915505 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5da7fc640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd5dc0a5780 0x7fd5dc1cc790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57284/0 (socket says 172.21.15.7:57284) 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1775284992 0 0) 0x7fd5dc0a43c0 con 0x7fd5dc13e310 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): my global_id is 74265 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5c8003520 con 0x7fd5dc13e310 2024-05-08T15:47:11.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325907144 0 0) 0x7fd5dc0a1e00 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): my global_id is 4261 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5dc0a43c0 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3487290536 0 0) 0x7fd5dc1ccea0 con 0x7fd5dc0a5780 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 10 monclient(hunting): my global_id is 4293 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5dc0a1e00 con 0x7fd5dc0a5780 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.985+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2165075585 0 0) 0x7fd5dc0a43c0 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5dc1ccea0 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd5cc0012d0 con 0x7fd5dc0a53a0 2024-05-08T15:47:11.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): dump: 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.997 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1892644030 0 0) 0x7fd5c8003520 con 0x7fd5dc13e310 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5dc0a43c0 con 0x7fd5dc13e310 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3255156230 0 0) 0x7fd5dc0a1e00 con 0x7fd5dc0a5780 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd5c8003520 con 0x7fd5dc0a5780 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd5d4001960 con 0x7fd5dc13e310 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): dump: 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:11.998 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd5d0001450 con 0x7fd5dc0a5780 2024-05-08T15:47:11.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient(hunting): dump: 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.000 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 514213080 0 0) 0x7fd5dc1ccea0 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 >> v1:172.21.15.7:6792/0 conn(0x7fd5dc13e310 legacy=0x7fd5dc1d45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 >> v1:172.21.15.7:6789/0 conn(0x7fd5dc0a5780 legacy=0x7fd5dc1cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 monclient: found mon.c 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5dc135f00 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5e194b640 5 monclient: authenticate success, global_id 4261 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5e194b640 10 monclient: _renew_subs 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5e194b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd5dc1373b0 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5e194b640 10 monclient: _renew_subs 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5e194b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fd5dc138660 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd5cc0015d0 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd5cc004dd0 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5d97fa640 4 set_mon_vals no callback set 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.986+0000 7fd5c3fff640 10 monclient: dump: 2024-05-08T15:47:12.002 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.003 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.987+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd5cc010150 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.987+0000 7fd5e194b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.987+0000 7fd5e194b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.987+0000 7fd5e194b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.987+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc005080 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.987+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(17..17 src has 1..17) ==== 3506+0+0 (unknown 993385953 0 0) 0x7fd5cc046bc0 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.987+0000 7fd5da7fc640 10 monclient: get_auth_request con 0x7fd5c803a420 auth_method 0 2024-05-08T15:47:12.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.992+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fd5cc015e20 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.992+0000 7fd5c3fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:11.992+0000 7fd5c3fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:12.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.178+0000 7fd5e194b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:12.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.178+0000 7fd5e194b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:12.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.178+0000 7fd5e194b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.178+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5bc005370 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.179+0000 7fd5c3fff640 1 -- 172.21.15.7:0/1274915505 <== mon.4 v1:172.21.15.7:6791/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+10735 (unknown 3338201251 0 1285910653) 0x7fd5cc015720 con 0x7fd5dc0a53a0 2024-05-08T15:47:12.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.179+0000 7fd5c3fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:12.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.179+0000 7fd5c3fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:12.176 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:47:12.166909+0000","last_up_change":"2024-05-08T15:47:03.939015+0000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T15:47:03.139226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T15:47:09.761977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6804","nonce":3231689356}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6805","nonce":3231689356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6807","nonce":3231689356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6806","nonce":3231689356}]},"public_addr":"172.21.15.7:6804/3231689356","cluster_addr":"172.21.15.7:6805/3231689356","heartbeat_back_addr":"172.21.15.7:6807/3231689356","heartbeat_front_addr":"172.21.15.7:6806/3231689356","state":["exists","up"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6800","nonce":601836183}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6801","nonce":601836183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6803","nonce":601836183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6802","nonce":601836183}]},"public_addr":"172.21.15.7:6800/601836183","cluster_addr":"172.21.15.7:6801/601836183","heartbeat_back_addr":"172.21.15.7:6803/601836183","heartbeat_front_addr":"172.21.15.7:6802/601836183","state":["exists","up"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6808","nonce":2598303896}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6809","nonce":2598303896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6811","nonce":2598303896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6810","nonce":2598303896}]},"public_addr":"172.21.15.7:6808/2598303896","cluster_addr":"172.21.15.7:6809/2598303896","heartbeat_back_addr":"172.21.15.7:6811/2598303896","heartbeat_front_addr":"172.21.15.7:6810/2598303896","state":["exists","up"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6802","nonce":1326537731}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6805","nonce":1326537731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6807","nonce":1326537731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6806","nonce":1326537731}]},"public_addr":"172.21.15.81:6802/1326537731","cluster_addr":"172.21.15.81:6805/1326537731","heartbeat_back_addr":"172.21.15.81:6807/1326537731","heartbeat_front_addr":"172.21.15.81:6806/1326537731","state":["exists","up"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6808","nonce":193838907}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6809","nonce":193838907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6811","nonce":193838907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6810","nonce":193838907}]},"public_addr":"172.21.15.81:6808/193838907","cluster_addr":"172.21.15.81:6809/193838907","heartbeat_back_addr":"172.21.15.81:6811/193838907","heartbeat_front_addr":"172.21.15.81:6810/193838907","state":["exists","up"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6800","nonce":385867939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6801","nonce":385867939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6804","nonce":385867939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6803","nonce":385867939}]},"public_addr":"172.21.15.81:6800/385867939","cluster_addr":"172.21.15.81:6801/385867939","heartbeat_back_addr":"172.21.15.81:6804/385867939","heartbeat_front_addr":"172.21.15.81:6803/385867939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.702217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.485746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.746567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.749309+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.745027+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.779789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 >> v1:172.21.15.81:6812/847627118 conn(0x7fd5c803a420 legacy=0x7fd5c803c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 10 monclient: shutdown 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 >> v1:172.21.15.7:6791/0 conn(0x7fd5dc0a53a0 legacy=0x7fd5dc1cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 shutdown_connections 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 >> 172.21.15.7:0/1274915505 conn(0x7fd5dc0a9fa0 msgr2=0x7fd5dc0a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 shutdown_connections 2024-05-08T15:47:12.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.181+0000 7fd5e194b640 1 -- 172.21.15.7:0/1274915505 wait complete. 2024-05-08T15:47:12.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T15:47:12.189 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T15:47:12.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T15:47:12.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T15:47:12.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T15:47:12.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T15:47:12.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.312+0000 7f086a60f640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.312+0000 7f086a60f640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.312+0000 7f086a60f640 10 monclient: monmap: 2024-05-08T15:47:12.310 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.310 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.310 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.313624+0000 2024-05-08T15:47:12.310 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.313624+0000 2024-05-08T15:47:12.310 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.311 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.312 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.312 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.313+0000 7f086a60f640 1 Processor -- start 2024-05-08T15:47:12.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.313+0000 7f086a60f640 1 -- start start 2024-05-08T15:47:12.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.313+0000 7f086a60f640 10 monclient: init 2024-05-08T15:47:12.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 10 monclient: _add_conns ranks=[3,8,0,5,2,6,7,4,1] 2024-05-08T15:47:12.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 10 monclient(hunting): picked mon.noname-g con 0x7f08640a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 10 monclient(hunting): picked mon.noname-e con 0x7f08640a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 10 monclient(hunting): picked mon.noname-a con 0x7f086413e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08640a1660 con 0x7f086413e310 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f08637fe640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f08640a5780 0x7f086413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54486/0 (socket says 172.21.15.7:54486) 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f08637fe640 1 -- 172.21.15.7:0/651582184 learned_addr learned my addr 172.21.15.7:0/651582184 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 1 -- 172.21.15.7:0/651582184 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08640a28f0 con 0x7f08640a53a0 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 1 -- 172.21.15.7:0/651582184 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08640a2bd0 con 0x7f08640a5780 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7f086a60f640 10 monclient(hunting): authenticate will time out at 840.429351s 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465600839 0 0) 0x7f08640a28f0 con 0x7f08640a53a0 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7fc91c48c640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7fc91c48c640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.314+0000 7fc91c48c640 10 monclient: monmap: 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.315380+0000 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.315380+0000 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.315 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.316 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.316 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.316 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): my global_id is 74226 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f084c003520 con 0x7f08640a53a0 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1968792268 0 0) 0x7f08640a1660 con 0x7f086413e310 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): my global_id is 4302 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08640a28f0 con 0x7f086413e310 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866430913 0 0) 0x7f08640a2bd0 con 0x7f08640a5780 2024-05-08T15:47:12.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient(hunting): my global_id is 4256 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08640a1660 con 0x7f08640a5780 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 612840214 0 0) 0x7f084c003520 con 0x7f08640a53a0 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 >> v1:172.21.15.7:6793/0 conn(0x7f08640a5780 legacy=0x7f086413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 >> v1:172.21.15.7:6789/0 conn(0x7f086413e310 legacy=0x7f0864140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 monclient: found mon.noname-g 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08640a6730 con 0x7f08640a53a0 2024-05-08T15:47:12.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7fc91c48c640 1 Processor -- start 2024-05-08T15:47:12.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08540017b0 con 0x7f08640a53a0 2024-05-08T15:47:12.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f086a60f640 5 monclient: authenticate success, global_id 74226 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f086a60f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7f0862ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 10 monclient: dump: 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.321 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f086a60f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7fc91c48c640 1 -- start start 2024-05-08T15:47:12.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.315+0000 7fc91c48c640 10 monclient: init 2024-05-08T15:47:12.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0854001ad0 con 0x7f08640a53a0 2024-05-08T15:47:12.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f086a60f640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f086a60f640 4 set_mon_vals no callback set 2024-05-08T15:47:12.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 1 -- 172.21.15.7:0/651582184 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0854003950 con 0x7f08640a53a0 2024-05-08T15:47:12.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7f0862ffd640 10 monclient: dump: 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.324 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.325 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7fc91c48c640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7fc91c48c640 10 monclient: _add_conns ranks=[3,2,0,4,6,7,8,1,5] 2024-05-08T15:47:12.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.316+0000 7fc91c48c640 10 monclient(hunting): picked mon.noname-g con 0x7fc9140a1580 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc91c48c640 10 monclient(hunting): picked mon.noname-b con 0x7fc914130a80 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7f086a60f640 10 monclient: shutdown 2024-05-08T15:47:12.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7f086a60f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7f086a60f640 1 -- 172.21.15.7:0/651582184 >> v1:172.21.15.81:6790/0 conn(0x7f08640a53a0 legacy=0x7f08640a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919a27640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fc914130a80 0x7fc914138b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44142/0 (socket says 172.21.15.7:44142) 2024-05-08T15:47:12.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919a27640 1 -- 172.21.15.7:0/939910381 learned_addr learned my addr 172.21.15.7:0/939910381 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7f086a60f640 1 -- 172.21.15.7:0/651582184 shutdown_connections 2024-05-08T15:47:12.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7f086a60f640 1 -- 172.21.15.7:0/651582184 >> 172.21.15.7:0/651582184 conn(0x7f08640a9fa0 msgr2=0x7f08640aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc91c48c640 10 monclient(hunting): picked mon.noname-a con 0x7fc91413a2b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc91406f950 con 0x7fc91413a2b0 2024-05-08T15:47:12.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc914131140 con 0x7fc914130a80 2024-05-08T15:47:12.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7f086a60f640 1 -- 172.21.15.7:0/651582184 shutdown_connections 2024-05-08T15:47:12.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7f086a60f640 1 -- 172.21.15.7:0/651582184 wait complete. 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc914070740 con 0x7fc9140a1580 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc91c48c640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc91c48c640 10 monclient(hunting): authenticate will time out at 840.432096s 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4261997846 0 0) 0x7fc91406f950 con 0x7fc91413a2b0 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): my global_id is 4311 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc900003520 con 0x7fc91413a2b0 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 626673797 0 0) 0x7fc914131140 con 0x7fc914130a80 2024-05-08T15:47:12.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 10 monclient(hunting): my global_id is 74234 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc91406f950 con 0x7fc914130a80 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3032606650 0 0) 0x7fc91406f950 con 0x7fc914130a80 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.317+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 >> v1:172.21.15.81:6790/0 conn(0x7fc9140a1580 legacy=0x7fc914072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 >> v1:172.21.15.7:6789/0 conn(0x7fc91413a2b0 legacy=0x7fc91413c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 1 monclient: found mon.noname-b 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc914070a20 con 0x7fc914130a80 2024-05-08T15:47:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc904001790 con 0x7fc914130a80 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: dump: 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.332 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc91c48c640 5 monclient: authenticate success, global_id 74234 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc91c48c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc9040019f0 con 0x7fc914130a80 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc91c48c640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 1 -- 172.21.15.7:0/939910381 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc904001be0 con 0x7fc914130a80 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc919226640 10 monclient: dump: 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.334 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7fc91c48c640 4 set_mon_vals no callback set 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7f086a60f640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.318+0000 7f086a60f640 10 monclient: monmap: 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.335 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.336 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 10 monclient: shutdown 2024-05-08T15:47:12.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 >> v1:172.21.15.7:6790/0 conn(0x7fc914130a80 legacy=0x7fc914138b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7f086a60f640 1 Processor -- start 2024-05-08T15:47:12.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 shutdown_connections 2024-05-08T15:47:12.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 >> 172.21.15.7:0/939910381 conn(0x7fc9140a6280 msgr2=0x7fc9140a2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 shutdown_connections 2024-05-08T15:47:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7f086a60f640 1 -- start start 2024-05-08T15:47:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 1 -- 172.21.15.7:0/939910381 wait complete. 2024-05-08T15:47:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7f086a60f640 10 monclient: init 2024-05-08T15:47:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.319+0000 7fc91c48c640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 10 monclient: _add_conns ranks=[1,7,0,8,2,5,6,4,3] 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7fc91c48c640 10 monclient: monmap: 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.340 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.341 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 10 monclient(hunting): picked mon.noname-f con 0x7f08640a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 10 monclient(hunting): picked mon.noname-i con 0x7f08640a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 10 monclient(hunting): picked mon.noname-a con 0x7f086413e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0864135110 con 0x7f086413e310 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0864134a10 con 0x7f08640a53a0 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08640a43c0 con 0x7f08640a5780 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f086a60f640 10 monclient(hunting): authenticate will time out at 840.434829s 2024-05-08T15:47:12.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0868bac640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f086413e310 0x7f08641d4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57320/0 (socket says 172.21.15.7:57320) 2024-05-08T15:47:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0868bac640 1 -- 172.21.15.7:0/1216877592 learned_addr learned my addr 172.21.15.7:0/1216877592 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7fc91c48c640 1 Processor -- start 2024-05-08T15:47:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7fc91c48c640 1 -- start start 2024-05-08T15:47:12.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7fc91c48c640 10 monclient: init 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313365735 0 0) 0x7f08640a28f0 con 0x7f086413e310 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): my global_id is 4320 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0850003520 con 0x7f086413e310 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 589420651 0 0) 0x7f0864134a10 con 0x7f08640a53a0 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): my global_id is 4249 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08640a28f0 con 0x7f08640a53a0 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3201385194 0 0) 0x7f08640a43c0 con 0x7f08640a5780 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 10 monclient(hunting): my global_id is 4264 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.320+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0864134a10 con 0x7f08640a5780 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2600152429 0 0) 0x7f08640a28f0 con 0x7f08640a53a0 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08640a43c0 con 0x7f08640a53a0 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0854003a10 con 0x7f08640a53a0 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:12.346 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.347 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2058274420 0 0) 0x7f0864134a10 con 0x7f08640a5780 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08640a28f0 con 0x7f08640a5780 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0858001900 con 0x7f08640a5780 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.348 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1845292894 0 0) 0x7f0850003520 con 0x7f086413e310 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 10 monclient: _add_conns ranks=[6,5,2,4,8,3,7,1,0] 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 10 monclient(hunting): picked mon.noname-d con 0x7fc9140a1580 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 10 monclient(hunting): picked mon.noname-h con 0x7fc914130a80 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91a228640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fc9140a1580 0x7fc914129530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60308/0 (socket says 172.21.15.7:60308) 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 10 monclient(hunting): picked mon.noname-b con 0x7fc91413a2b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91aa29640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fc91413a2b0 0x7fc9141d0fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44154/0 (socket says 172.21.15.7:44154) 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91aa29640 1 -- 172.21.15.7:0/2267887045 learned_addr learned my addr 172.21.15.7:0/2267887045 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9141390e0 con 0x7fc91413a2b0 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc914136550 con 0x7fc914130a80 2024-05-08T15:47:12.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc914134da0 con 0x7fc9140a1580 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0864134a10 con 0x7f086413e310 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f085c001c90 con 0x7f086413e310 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.351 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.352 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc91c48c640 10 monclient(hunting): authenticate will time out at 840.436355s 2024-05-08T15:47:12.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 819504729 0 0) 0x7fc9141390e0 con 0x7fc91413a2b0 2024-05-08T15:47:12.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc90affd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1268305932 0 0) 0x7f08640a43c0 con 0x7f08640a53a0 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 >> v1:172.21.15.81:6792/0 conn(0x7f08640a5780 legacy=0x7f086413c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 >> v1:172.21.15.7:6789/0 conn(0x7f086413e310 legacy=0x7f08641d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 monclient: found mon.f 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08640a1e00 con 0x7f08640a53a0 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7f0860ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f086a60f640 5 monclient: authenticate success, global_id 4249 2024-05-08T15:47:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f086a60f640 10 monclient: _renew_subs 2024-05-08T15:47:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f086a60f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f086a60f640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f086413cb10 con 0x7f08640a53a0 2024-05-08T15:47:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc90affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f086a60f640 10 monclient: _renew_subs 2024-05-08T15:47:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f086a60f640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f086a60f640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f08641330e0 con 0x7f08640a53a0 2024-05-08T15:47:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc90affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc90affd640 10 monclient(hunting): my global_id is 74243 2024-05-08T15:47:12.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.321+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8f8003520 con 0x7fc91413a2b0 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2827403742 0 0) 0x7fc8f8003520 con 0x7fc91413a2b0 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc9141390e0 con 0x7fc91413a2b0 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0854001830 con 0x7f08640a53a0 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f0860ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0854001c80 con 0x7f08640a53a0 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f0860ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f0860ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f0860ff9640 10 monclient: dump: 2024-05-08T15:47:12.357 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.358 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7f08627fc640 4 set_mon_vals no callback set 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc90c001910 con 0x7fc91413a2b0 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient(hunting): dump: 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.360 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1894374559 0 0) 0x7fc9141390e0 con 0x7fc91413a2b0 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 >> v1:172.21.15.7:6792/0 conn(0x7fc9140a1580 legacy=0x7fc914129530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 >> v1:172.21.15.81:6791/0 conn(0x7fc914130a80 legacy=0x7fc9141cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 monclient: found mon.b 2024-05-08T15:47:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc914137100 con 0x7fc91413a2b0 2024-05-08T15:47:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.323+0000 7f084a7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:12.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.323+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.81:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f0864067750 con 0x7f08640a53a0 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc91c48c640 5 monclient: authenticate success, global_id 74243 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc91c48c640 10 monclient: _renew_subs 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc91c48c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc914136860 con 0x7fc91413a2b0 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc91c48c640 10 monclient: _renew_subs 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc91c48c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fc914133cd0 con 0x7fc91413a2b0 2024-05-08T15:47:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc90c003c20 con 0x7fc91413a2b0 2024-05-08T15:47:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc90affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f085400f420 con 0x7f08640a53a0 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.322+0000 7fc918a25640 4 set_mon_vals no callback set 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.323+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc90c004110 con 0x7fc91413a2b0 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.323+0000 7fc90affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.323+0000 7fc90affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.323+0000 7fc90affd640 10 monclient: dump: 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.366 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.367 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.368 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fc90c004330 con 0x7fc91413a2b0 2024-05-08T15:47:12.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f0854045a40 con 0x7f08640a53a0 2024-05-08T15:47:12.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fc90c0471b0 con 0x7fc91413a2b0 2024-05-08T15:47:12.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7fc91a228640 10 monclient: get_auth_request con 0x7fc8f8039b20 auth_method 0 2024-05-08T15:47:12.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.7:6808/2598303896 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0854001ad0 con 0x7f08500410a0 2024-05-08T15:47:12.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (unknown 77724187 0 0) 0x7f0854045e40 con 0x7f08640a53a0 2024-05-08T15:47:12.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.324+0000 7f0860ff9640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-05-08T15:47:12.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.325+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.81:6800/385867939 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc8fc0053a0 con 0x7fc8fc001630 2024-05-08T15:47:12.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.325+0000 7f08637fe640 10 monclient: get_auth_request con 0x7f085003a560 auth_method 0 2024-05-08T15:47:12.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.325+0000 7f0868bac640 10 monclient: get_auth_request con 0x7f08500410a0 auth_method 0 2024-05-08T15:47:12.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.325+0000 7fc919a27640 10 monclient: get_auth_request con 0x7fc8fc001630 auth_method 0 2024-05-08T15:47:12.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.326+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== osd.2 v1:172.21.15.7:6808/2598303896 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f0854001ad0 con 0x7f08500410a0 2024-05-08T15:47:12.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.326+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== osd.5 v1:172.21.15.81:6800/385867939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fc8fc0053a0 con 0x7fc8fc001630 2024-05-08T15:47:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.354+0000 7f3208214640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.354+0000 7f3208214640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.354+0000 7f3208214640 10 monclient: monmap: 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.355525+0000 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.355525+0000 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.375 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.355+0000 7f3208214640 1 Processor -- start 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.355+0000 7f3208214640 1 -- start start 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.355+0000 7f3208214640 10 monclient: init 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.355+0000 7f3208214640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.355+0000 7f3208214640 10 monclient: _add_conns ranks=[8,6,5,4,2,1,0,3,7] 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.355+0000 7f3208214640 10 monclient(hunting): picked mon.noname-e con 0x7f32000a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3208214640 10 monclient(hunting): picked mon.noname-d con 0x7f32000a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3208214640 10 monclient(hunting): picked mon.noname-h con 0x7f320013e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3208214640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000a1660 con 0x7f320013e310 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3208214640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000a28f0 con 0x7f32000a5780 2024-05-08T15:47:12.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3208214640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000a2bd0 con 0x7f32000a53a0 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3208214640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3208214640 10 monclient(hunting): authenticate will time out at 840.470820s 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f32057af640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f32000a5780 0x7f320013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60312/0 (socket says 172.21.15.7:60312) 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f32057af640 1 -- 172.21.15.7:0/3457348275 learned_addr learned my addr 172.21.15.7:0/3457348275 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029149393 0 0) 0x7f32000a2bd0 con 0x7f32000a53a0 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): my global_id is 4265 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31ec003520 con 0x7f32000a53a0 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122506544 0 0) 0x7f32000a28f0 con 0x7f32000a5780 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): my global_id is 74274 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32000a2bd0 con 0x7f32000a5780 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1767844999 0 0) 0x7f32000a1660 con 0x7f320013e310 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 10 monclient(hunting): my global_id is 4289 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.356+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32000a28f0 con 0x7f320013e310 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1780232395 0 0) 0x7f31ec003520 con 0x7f32000a53a0 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 >> v1:172.21.15.7:6792/0 conn(0x7f32000a5780 legacy=0x7f320013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 >> v1:172.21.15.81:6791/0 conn(0x7f320013e310 legacy=0x7f3200140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 monclient: found mon.noname-e 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32000a6730 con 0x7f32000a53a0 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31fc0017b0 con 0x7f32000a53a0 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: dump: 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.379 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3208214640 5 monclient: authenticate success, global_id 4265 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3208214640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f31fc001a10 con 0x7f32000a53a0 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 1 -- 172.21.15.7:0/3457348275 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31fc003950 con 0x7f32000a53a0 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3208214640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3208214640 4 set_mon_vals no callback set 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3204fae640 10 monclient: dump: 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.381 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3208214640 10 monclient: shutdown 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3208214640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.357+0000 7f3208214640 1 -- 172.21.15.7:0/3457348275 >> v1:172.21.15.7:6793/0 conn(0x7f32000a53a0 legacy=0x7f32000a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.358+0000 7f3208214640 1 -- 172.21.15.7:0/3457348275 shutdown_connections 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.358+0000 7f3208214640 1 -- 172.21.15.7:0/3457348275 >> 172.21.15.7:0/3457348275 conn(0x7f32000a9fa0 msgr2=0x7f32000aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.358+0000 7f3208214640 1 -- 172.21.15.7:0/3457348275 shutdown_connections 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.358+0000 7f3208214640 1 -- 172.21.15.7:0/3457348275 wait complete. 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.358+0000 7f3208214640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.358+0000 7f3208214640 10 monclient: monmap: 2024-05-08T15:47:12.382 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.383 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 1 Processor -- start 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 1 -- start start 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient: init 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f84da9bd640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f84da9bd640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f84da9bd640 10 monclient: monmap: 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.384 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.360483+0000 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.360483+0000 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.385 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient: _add_conns ranks=[4,5,6,8,2,1,3,7,0] 2024-05-08T15:47:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient(hunting): picked mon.noname-c con 0x7f32000a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:12.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient(hunting): picked mon.noname-h con 0x7f32000a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3205fb0640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f32000a53a0 0x7f3200137ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51034/0 (socket says 172.21.15.7:51034) 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient(hunting): picked mon.noname-d con 0x7f320013e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3205fb0640 1 -- 172.21.15.7:0/439275249 learned_addr learned my addr 172.21.15.7:0/439275249 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000a43c0 con 0x7f32000a53a0 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000a1e00 con 0x7f32000a5780 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32001c9900 con 0x7f320013e310 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.359+0000 7f3208214640 10 monclient(hunting): authenticate will time out at 840.474464s 2024-05-08T15:47:12.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 --> v1:172.21.15.7:6808/2598303896 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0864075820 con 0x7f08500410a0 2024-05-08T15:47:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1590108745 0 0) 0x7f32001c9900 con 0x7f320013e310 2024-05-08T15:47:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): my global_id is 74283 2024-05-08T15:47:12.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31e4003520 con 0x7f320013e310 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182089858 0 0) 0x7f32000a43c0 con 0x7f32000a53a0 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): my global_id is 4270 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32001c9900 con 0x7f32000a53a0 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3860047917 0 0) 0x7f31e4003520 con 0x7f320013e310 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32000a43c0 con 0x7f320013e310 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31f8001d30 con 0x7f320013e310 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): dump: 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.389 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.390 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118116091 0 0) 0x7f32000a1e00 con 0x7f32000a5780 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): my global_id is 4298 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31e4003520 con 0x7f32000a5780 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3000414289 0 0) 0x7f32001c9900 con 0x7f32000a53a0 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f32000a1e00 con 0x7f32000a53a0 2024-05-08T15:47:12.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f0860ff9640 1 -- 172.21.15.7:0/1216877592 <== osd.2 v1:172.21.15.7:6808/2598303896 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3580782990) 0x7f0864075820 con 0x7f08500410a0 2024-05-08T15:47:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31fc0016b0 con 0x7f32000a53a0 2024-05-08T15:47:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient(hunting): dump: 2024-05-08T15:47:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.392 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.393 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4122189068 0 0) 0x7f32000a43c0 con 0x7f320013e310 2024-05-08T15:47:12.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 >> v1:172.21.15.81:6791/0 conn(0x7f32000a5780 legacy=0x7f32001d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 >> v1:172.21.15.7:6791/0 conn(0x7f32000a53a0 legacy=0x7f3200137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 monclient: found mon.d 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32001346a0 con 0x7f320013e310 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.360+0000 7f31f67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f3208214640 5 monclient: authenticate success, global_id 74283 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f3208214640 10 monclient: _renew_subs 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f3208214640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f32001385d0 con 0x7f320013e310 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f3208214640 10 monclient: _renew_subs 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f3208214640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f32001353e0 con 0x7f320013e310 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f31f8001ff0 con 0x7f320013e310 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f31f67fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f31f7fff640 4 set_mon_vals no callback set 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 >> v1:172.21.15.7:6808/2598303896 conn(0x7f08500410a0 legacy=0x7f08500434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 >> v1:172.21.15.81:6812/847627118 conn(0x7f085003a560 legacy=0x7f085003ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 10 monclient: shutdown 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 >> v1:172.21.15.81:6789/0 conn(0x7f08640a53a0 legacy=0x7f086413bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f0868bac640 1 -- 172.21.15.7:0/1216877592 reap_dead start 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f31f8003f80 con 0x7f320013e310 2024-05-08T15:47:12.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f31f67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f31f67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f31f67fc640 10 monclient: dump: 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.398 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 shutdown_connections 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 >> 172.21.15.7:0/1216877592 conn(0x7f08640a9fa0 msgr2=0x7f08640a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 shutdown_connections 2024-05-08T15:47:12.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7f084a7fc640 1 -- 172.21.15.7:0/1216877592 wait complete. 2024-05-08T15:47:12.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.361+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 --> v1:172.21.15.81:6800/385867939 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc8fc007030 con 0x7fc8fc001630 2024-05-08T15:47:12.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7f84da9bd640 1 Processor -- start 2024-05-08T15:47:12.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7fc90affd640 1 -- 172.21.15.7:0/2267887045 <== osd.5 v1:172.21.15.81:6800/385867939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 654699149) 0x7fc8fc007030 con 0x7fc8fc001630 2024-05-08T15:47:12.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7f3208214640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:12.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6792/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f3200075820 con 0x7f320013e310 2024-05-08T15:47:12.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7f84da9bd640 1 -- start start 2024-05-08T15:47:12.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7f84da9bd640 10 monclient: init 2024-05-08T15:47:12.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 >> v1:172.21.15.81:6800/385867939 conn(0x7fc8fc001630 legacy=0x7fc8fc003af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 >> v1:172.21.15.81:6812/847627118 conn(0x7fc8f8039b20 legacy=0x7fc8f803bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7fc91c48c640 10 monclient: shutdown 2024-05-08T15:47:12.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7fc91c48c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 >> v1:172.21.15.7:6790/0 conn(0x7fc91413a2b0 legacy=0x7fc9141d0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f31f800f300 con 0x7f320013e310 2024-05-08T15:47:12.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7fc91aa29640 1 -- 172.21.15.7:0/2267887045 reap_dead start 2024-05-08T15:47:12.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f84da9bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f84da9bd640 10 monclient: _add_conns ranks=[6,5,0,8,1,3,7,2,4] 2024-05-08T15:47:12.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 shutdown_connections 2024-05-08T15:47:12.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 >> 172.21.15.7:0/2267887045 conn(0x7fc9140a6280 msgr2=0x7fc9140a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f84da9bd640 10 monclient(hunting): picked mon.noname-d con 0x7f84d40a2700 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.362+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f31f8045df0 con 0x7f320013e310 2024-05-08T15:47:12.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6804/3231689356 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31e4044b80 con 0x7f31e4040f10 2024-05-08T15:47:12.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (unknown 77724187 0 0) 0x7f31f80461f0 con 0x7f320013e310 2024-05-08T15:47:12.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f31f67fc640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-05-08T15:47:12.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f32057af640 10 monclient: get_auth_request con 0x7f31e4040f10 auth_method 0 2024-05-08T15:47:12.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f3205fb0640 10 monclient: get_auth_request con 0x7f31e403a2c0 auth_method 0 2024-05-08T15:47:12.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 shutdown_connections 2024-05-08T15:47:12.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f84d3fff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f84d40a2700 0x7f84d40a2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60336/0 (socket says 172.21.15.7:60336) 2024-05-08T15:47:12.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7fc91c48c640 1 -- 172.21.15.7:0/2267887045 wait complete. 2024-05-08T15:47:12.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f84d3fff640 1 -- 172.21.15.7:0/2999175597 learned_addr learned my addr 172.21.15.7:0/2999175597 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f2c0bbdb640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f2c0bbdb640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f2c0bbdb640 10 monclient: monmap: 2024-05-08T15:47:12.406 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.364645+0000 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.364645+0000 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.407 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f84da9bd640 10 monclient(hunting): picked mon.noname-h con 0x7f84d4072710 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.363+0000 7f84da9bd640 10 monclient(hunting): picked mon.noname-a con 0x7f84d40a31f0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d409fd50 con 0x7f84d40a31f0 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d40a0030 con 0x7f84d4072710 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d40a4230 con 0x7f84d40a2700 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84da9bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84da9bd640 10 monclient(hunting): authenticate will time out at 840.479035s 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2003337239 0 0) 0x7f84d409fd50 con 0x7f84d40a31f0 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): my global_id is 4329 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84bc003520 con 0x7f84d40a31f0 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186112002 0 0) 0x7f84d40a4230 con 0x7f84d40a2700 2024-05-08T15:47:12.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): my global_id is 74292 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84d409fd50 con 0x7f84d40a2700 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 438269722 0 0) 0x7f84d40a0030 con 0x7f84d4072710 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 10 monclient(hunting): my global_id is 4307 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84d40a4230 con 0x7f84d4072710 2024-05-08T15:47:12.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f2c0bbdb640 1 Processor -- start 2024-05-08T15:47:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== osd.0 v1:172.21.15.7:6804/3231689356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f31e4044b80 con 0x7f31e4040f10 2024-05-08T15:47:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2643877413 0 0) 0x7f84d40a4230 con 0x7f84d4072710 2024-05-08T15:47:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 >> v1:172.21.15.7:6792/0 conn(0x7f84d40a2700 legacy=0x7f84d40a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f2c0bbdb640 1 -- start start 2024-05-08T15:47:12.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.364+0000 7f2c0bbdb640 10 monclient: init 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 >> v1:172.21.15.7:6789/0 conn(0x7f84d40a31f0 legacy=0x7f84d413d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 monclient: found mon.noname-h 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d409e9f0 con 0x7f84d4072710 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f84c80016d0 con 0x7f84d4072710 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: dump: 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.413 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84da9bd640 5 monclient: authenticate success, global_id 4307 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84da9bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7fa161282640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7fa161282640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7fa161282640 10 monclient: monmap: 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.366589+0000 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.366589+0000 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.415 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 10 monclient: _add_conns ranks=[3,1,2,8,5,6,7,4,0] 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 10 monclient(hunting): picked mon.noname-g con 0x7f2c040a3f00 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 10 monclient(hunting): picked mon.noname-f con 0x7f2c040a2e10 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 10 monclient(hunting): picked mon.noname-b con 0x7f2c040a3220 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c0409ff60 con 0x7f2c040a2e10 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c0406b670 con 0x7f2c040a3220 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c0409ec90 con 0x7f2c040a3f00 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0bbdb640 10 monclient(hunting): authenticate will time out at 840.480265s 2024-05-08T15:47:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f84c8001930 con 0x7f84d4072710 2024-05-08T15:47:12.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 1 -- 172.21.15.7:0/2999175597 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f84c80035f0 con 0x7f84d4072710 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f84d2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f84d2ffd640 10 monclient: dump: 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.418 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0a178640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f2c040a3220 0x7f2c041449f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44162/0 (socket says 172.21.15.7:44162) 2024-05-08T15:47:12.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.365+0000 7f2c0a178640 1 -- 172.21.15.7:0/1848384605 learned_addr learned my addr 172.21.15.7:0/1848384605 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f84da9bd640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f84da9bd640 4 set_mon_vals no callback set 2024-05-08T15:47:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1332502002 0 0) 0x7f2c0409ff60 con 0x7f2c040a2e10 2024-05-08T15:47:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): my global_id is 4258 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bf0003520 con 0x7f2c040a2e10 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3254972200 0 0) 0x7f2c0406b670 con 0x7f2c040a3220 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient(hunting): my global_id is 74252 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bf00048e0 con 0x7f2c040a3220 2024-05-08T15:47:12.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7fa161282640 1 Processor -- start 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 382450689 0 0) 0x7f2bf0003520 con 0x7f2c040a2e10 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 >> v1:172.21.15.81:6790/0 conn(0x7f2c040a3f00 legacy=0x7f2c040a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 >> v1:172.21.15.7:6790/0 conn(0x7f2c040a3220 legacy=0x7f2c041449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 monclient: found mon.noname-f 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0409e1f0 con 0x7f2c040a2e10 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c0bbdb640 5 monclient: authenticate success, global_id 4258 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c0bbdb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bf4001790 con 0x7f2c040a2e10 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: dump: 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.423 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c0bbdb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7fa161282640 1 -- start start 2024-05-08T15:47:12.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7fa161282640 10 monclient: init 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2bf4001ab0 con 0x7f2c040a2e10 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c0bbdb640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c0bbdb640 4 set_mon_vals no callback set 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 1 -- 172.21.15.7:0/1848384605 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bf4001790 con 0x7f2c040a2e10 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.366+0000 7f2c08975640 10 monclient: dump: 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.425 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.426 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f84da9bd640 10 monclient: shutdown 2024-05-08T15:47:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f84da9bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 >> v1:172.21.15.81:6791/0 conn(0x7f84d4072710 legacy=0x7f84d4139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 shutdown_connections 2024-05-08T15:47:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 >> 172.21.15.7:0/2999175597 conn(0x7f84d40a7320 msgr2=0x7f84d40a37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 shutdown_connections 2024-05-08T15:47:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 10 monclient: _add_conns ranks=[7,6,3,8,1,2,5,4,0] 2024-05-08T15:47:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f84da9bd640 1 -- 172.21.15.7:0/2999175597 wait complete. 2024-05-08T15:47:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 10 monclient(hunting): picked mon.noname-i con 0x7fa15c134af0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 10 monclient(hunting): picked mon.noname-d con 0x7fa15c0a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 10 monclient(hunting): picked mon.noname-g con 0x7fa15c13e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c0a6770 con 0x7fa15c13e310 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c0a1660 con 0x7fa15c0a49a0 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c0a2910 con 0x7fa15c134af0 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa161282640 10 monclient(hunting): authenticate will time out at 840.482254s 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa15a7fc640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fa15c0a49a0 0x7fa15c13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60340/0 (socket says 172.21.15.7:60340) 2024-05-08T15:47:12.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa15a7fc640 1 -- 172.21.15.7:0/470851964 learned_addr learned my addr 172.21.15.7:0/470851964 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f2c0bbdb640 10 monclient: shutdown 2024-05-08T15:47:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f2c0bbdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/1848384605 >> v1:172.21.15.81:6789/0 conn(0x7f2c040a2e10 legacy=0x7f2c041412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7fa15affd640 1 --1- 172.21.15.7:0/470851964 >> v1:172.21.15.81:6792/0 conn(0x7fa15c134af0 0x7fa15c134ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:33032/0 (socket says 172.21.15.7:33032) 2024-05-08T15:47:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/1848384605 shutdown_connections 2024-05-08T15:47:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.367+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/1848384605 >> 172.21.15.7:0/1848384605 conn(0x7f2c040a7320 msgr2=0x7f2c040a37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/1848384605 shutdown_connections 2024-05-08T15:47:12.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/1848384605 wait complete. 2024-05-08T15:47:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7f84da9bd640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3587318203 0 0) 0x7fa15c0a2910 con 0x7fa15c134af0 2024-05-08T15:47:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): my global_id is 4273 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa148003520 con 0x7fa15c134af0 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1993681636 0 0) 0x7fa15c0a6770 con 0x7fa15c13e310 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient(hunting): my global_id is 74253 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa15c0a2910 con 0x7fa15c13e310 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7f84da9bd640 10 monclient: monmap: 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.434 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.435 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 496015951 0 0) 0x7fa15c0a2910 con 0x7fa15c13e310 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 >> v1:172.21.15.81:6792/0 conn(0x7fa15c134af0 legacy=0x7fa15c134ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 >> v1:172.21.15.7:6792/0 conn(0x7fa15c0a49a0 legacy=0x7fa15c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 monclient: found mon.noname-g 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c0a2bf0 con 0x7fa15c13e310 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa154001b10 con 0x7fa15c13e310 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7fa159ffb640 10 monclient: dump: 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.437 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7f2c0bbdb640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.368+0000 7f2c0bbdb640 10 monclient: monmap: 2024-05-08T15:47:12.438 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.439 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa161282640 5 monclient: authenticate success, global_id 74253 2024-05-08T15:47:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa161282640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa154001d70 con 0x7fa15c13e310 2024-05-08T15:47:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa159ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa159ffb640 1 -- 172.21.15.7:0/470851964 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa154001f60 con 0x7fa15c13e310 2024-05-08T15:47:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa159ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa159ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7f84da9bd640 1 Processor -- start 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7f84da9bd640 1 -- start start 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7f84da9bd640 10 monclient: init 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa159ffb640 10 monclient: dump: 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.441 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7f2c0bbdb640 1 Processor -- start 2024-05-08T15:47:12.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7f2c0bbdb640 1 -- start start 2024-05-08T15:47:12.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7f2c0bbdb640 10 monclient: init 2024-05-08T15:47:12.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa161282640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.369+0000 7fa161282640 4 set_mon_vals no callback set 2024-05-08T15:47:12.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 10 monclient: _add_conns ranks=[2,3,1,5,7,6,8,0,4] 2024-05-08T15:47:12.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 10 monclient(hunting): picked mon.noname-b con 0x7f2c040a2e10 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7fa161282640 10 monclient: shutdown 2024-05-08T15:47:12.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7fa161282640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7fa161282640 1 -- 172.21.15.7:0/470851964 >> v1:172.21.15.81:6790/0 conn(0x7fa15c13e310 legacy=0x7fa15c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 10 monclient(hunting): picked mon.noname-g con 0x7f2c040a3220 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 10 monclient(hunting): picked mon.noname-f con 0x7f2c040a3f00 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c041d1700 con 0x7f2c040a3f00 2024-05-08T15:47:12.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7fa161282640 1 -- 172.21.15.7:0/470851964 shutdown_connections 2024-05-08T15:47:12.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7fa161282640 1 -- 172.21.15.7:0/470851964 >> 172.21.15.7:0/470851964 conn(0x7fa15c0a9fa0 msgr2=0x7fa15c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f84da9bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f84da9bd640 10 monclient: _add_conns ranks=[2,0,8,6,7,1,3,4,5] 2024-05-08T15:47:12.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c041b0f60 con 0x7f2c040a2e10 2024-05-08T15:47:12.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7fa161282640 1 -- 172.21.15.7:0/470851964 shutdown_connections 2024-05-08T15:47:12.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c0409e480 con 0x7f2c040a3220 2024-05-08T15:47:12.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7fa161282640 1 -- 172.21.15.7:0/470851964 wait complete. 2024-05-08T15:47:12.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0bbdb640 10 monclient(hunting): authenticate will time out at 840.484849s 2024-05-08T15:47:12.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0a178640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2c040a3f00 0x7f2c041d92e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:53356/0 (socket says 172.21.15.7:53356) 2024-05-08T15:47:12.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f84d3fff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f84d4072710 0x7f84d41bccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44180/0 (socket says 172.21.15.7:44180) 2024-05-08T15:47:12.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f2c0a178640 1 -- 172.21.15.7:0/3469258019 learned_addr learned my addr 172.21.15.7:0/3469258019 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.370+0000 7f84d3fff640 1 -- 172.21.15.7:0/721929879 learned_addr learned my addr 172.21.15.7:0/721929879 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 508781948 0 0) 0x7f2c041b0f60 con 0x7f2c040a2e10 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): my global_id is 74261 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2be8003520 con 0x7f2c040a2e10 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1940222717 0 0) 0x7f2c0409e480 con 0x7f2c040a3220 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7fa161282640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7fa161282640 10 monclient: monmap: 2024-05-08T15:47:12.450 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.450 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.450 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.451 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): my global_id is 74262 2024-05-08T15:47:12.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c041b0f60 con 0x7f2c040a3220 2024-05-08T15:47:12.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1913424859 0 0) 0x7f2be8003520 con 0x7f2c040a2e10 2024-05-08T15:47:12.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2c0409e480 con 0x7f2c040a2e10 2024-05-08T15:47:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2c000019a0 con 0x7f2c040a2e10 2024-05-08T15:47:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f84da9bd640 10 monclient(hunting): picked mon.noname-b con 0x7f84d4072710 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f84da9bd640 10 monclient(hunting): picked mon.noname-a con 0x7f84d40a2700 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f84da9bd640 10 monclient(hunting): picked mon.noname-e con 0x7f84d40a31f0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f84da9bd640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d41bd3e0 con 0x7f84d40a2700 2024-05-08T15:47:12.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): dump: 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.454 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.455 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.455 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.455 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3264606303 0 0) 0x7f84d41bd3e0 con 0x7f84d40a2700 2024-05-08T15:47:12.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 1 Processor -- start 2024-05-08T15:47:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 1 -- start start 2024-05-08T15:47:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient: init 2024-05-08T15:47:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f84da9bd640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d4072b60 con 0x7f84d4072710 2024-05-08T15:47:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient: _add_conns ranks=[4,1,0,3,2,8,7,5,6] 2024-05-08T15:47:12.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f84da9bd640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84d41a8350 con 0x7f84d40a31f0 2024-05-08T15:47:12.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f84da9bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f84da9bd640 10 monclient(hunting): authenticate will time out at 840.487551s 2024-05-08T15:47:12.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1483851897 0 0) 0x7f2c0409e480 con 0x7f2c040a2e10 2024-05-08T15:47:12.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 >> v1:172.21.15.81:6790/0 conn(0x7f2c040a3220 legacy=0x7f2c041d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 >> v1:172.21.15.81:6789/0 conn(0x7f2c040a3f00 legacy=0x7f2c041d92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 monclient: found mon.b 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0409ddc0 con 0x7f2c040a2e10 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2bfa7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2c0bbdb640 5 monclient: authenticate success, global_id 74261 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2c0bbdb640 10 monclient: _renew_subs 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2c0bbdb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): my global_id is 4338 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84c0003520 con 0x7f84d40a2700 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.371+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2c041b1520 con 0x7f2c040a2e10 2024-05-08T15:47:12.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f2c0bbdb640 10 monclient: _renew_subs 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f2c0bbdb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2c041207a0 con 0x7f2c040a2e10 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient(hunting): picked mon.noname-c con 0x7fa15c0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient(hunting): picked mon.noname-f con 0x7fa15c134af0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient(hunting): picked mon.noname-a con 0x7fa15c13e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c0a1e40 con 0x7fa15c13e310 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c0a0ef0 con 0x7fa15c134af0 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa15c137410 con 0x7fa15c0a49a0 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.372+0000 7fa161282640 10 monclient(hunting): authenticate will time out at 840.487583s 2024-05-08T15:47:12.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa15affd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fa15c0a49a0 0x7fa15c13ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51038/0 (socket says 172.21.15.7:51038) 2024-05-08T15:47:12.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa15affd640 1 -- 172.21.15.7:0/828734723 learned_addr learned my addr 172.21.15.7:0/828734723 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2c00001cc0 con 0x7f2c040a2e10 2024-05-08T15:47:12.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2c0bbdb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.7:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f2c041b1760 con 0x7f2c040a2e10 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2bfa7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2c00003be0 con 0x7f2c040a2e10 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 705711893 0 0) 0x7f84c0003520 con 0x7f84d40a2700 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f84d41bd3e0 con 0x7f84d40a2700 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f84c8001610 con 0x7f84d40a2700 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:12.464 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.465 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f84d0ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2291435232 0 0) 0x7fa15c137410 con 0x7fa15c0a49a0 2024-05-08T15:47:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa1537fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa1537fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa1537fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa1537fe640 10 monclient(hunting): my global_id is 4279 2024-05-08T15:47:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa13c003520 con 0x7fa15c0a49a0 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2bfa7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2bfa7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2bfa7fc640 10 monclient: dump: 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 771971033 0 0) 0x7fa13c003520 con 0x7fa15c0a49a0 2024-05-08T15:47:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa15c137410 con 0x7fa15c0a49a0 2024-05-08T15:47:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3304270358 0 0) 0x7f84d41a8350 con 0x7f84d40a31f0 2024-05-08T15:47:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient(hunting): my global_id is 4274 2024-05-08T15:47:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84c0003520 con 0x7f84d40a31f0 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa144001b50 con 0x7fa15c0a49a0 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): dump: 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 295325891 0 0) 0x7f84d41bd3e0 con 0x7f84d40a2700 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 >> v1:172.21.15.7:6793/0 conn(0x7f84d40a31f0 legacy=0x7f84d41d5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 >> v1:172.21.15.7:6790/0 conn(0x7f84d4072710 legacy=0x7f84d41bccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 1 monclient: found mon.a 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d41a9e10 con 0x7f84d40a2700 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038431234 0 0) 0x7fa15c0a2bf0 con 0x7fa15c13e310 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient(hunting): my global_id is 4347 2024-05-08T15:47:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa13c003520 con 0x7fa15c13e310 2024-05-08T15:47:12.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.373+0000 7f2bfbfff640 4 set_mon_vals no callback set 2024-05-08T15:47:12.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84da9bd640 5 monclient: authenticate success, global_id 4338 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84da9bd640 10 monclient: _renew_subs 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84da9bd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84da9bd640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f84d41aa6b0 con 0x7f84d40a2700 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84da9bd640 10 monclient: _renew_subs 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84da9bd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3737355745 0 0) 0x7fa15c137410 con 0x7fa15c0a49a0 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 >> v1:172.21.15.81:6789/0 conn(0x7fa15c134af0 legacy=0x7fa15c1d1cf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 >> v1:172.21.15.7:6789/0 conn(0x7fa15c13e310 legacy=0x7fa15c1d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 monclient: found mon.c 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c1cd600 con 0x7fa15c0a49a0 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f84da9bd640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f84d41a92b0 con 0x7f84d40a2700 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa144004b80 con 0x7fa15c0a49a0 2024-05-08T15:47:12.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2c00003e40 con 0x7f2c040a2e10 2024-05-08T15:47:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa144005070 con 0x7fa15c0a49a0 2024-05-08T15:47:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.374+0000 7fa1537fe640 10 monclient: dump: 2024-05-08T15:47:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa161282640 5 monclient: authenticate success, global_id 4279 2024-05-08T15:47:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa161282640 10 monclient: _renew_subs 2024-05-08T15:47:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa161282640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c139040 con 0x7fa15c0a49a0 2024-05-08T15:47:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f2c00004400 con 0x7f2c040a2e10 2024-05-08T15:47:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.81:6808/193838907 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2be8044d60 con 0x7f2be8041050 2024-05-08T15:47:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f84c8003b60 con 0x7f84d40a2700 2024-05-08T15:47:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f84d0ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f2c09176640 10 monclient: get_auth_request con 0x7f2be8039c30 auth_method 0 2024-05-08T15:47:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f84c8004050 con 0x7f84d40a2700 2024-05-08T15:47:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (unknown 77724187 0 0) 0x7f2c00011150 con 0x7f2c040a2e10 2024-05-08T15:47:12.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f2bfa7fc640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-05-08T15:47:12.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f2c0a178640 10 monclient: get_auth_request con 0x7f2be8041050 auth_method 0 2024-05-08T15:47:12.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f84d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f84d0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:12.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f84d0ff9640 10 monclient: dump: 2024-05-08T15:47:12.480 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.481 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.482 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa1597fa640 4 set_mon_vals no callback set 2024-05-08T15:47:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa161282640 10 monclient: _renew_subs 2024-05-08T15:47:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa161282640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fa15c1380c0 con 0x7fa15c0a49a0 2024-05-08T15:47:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.376+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f84c800f3d0 con 0x7f84d40a2700 2024-05-08T15:47:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.375+0000 7f84d27fc640 4 set_mon_vals no callback set 2024-05-08T15:47:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.376+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f84c8045be0 con 0x7f84d40a2700 2024-05-08T15:47:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6800/601836183 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84b40053a0 con 0x7f84b4001630 2024-05-08T15:47:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== osd.4 v1:172.21.15.81:6808/193838907 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f2be8044d60 con 0x7f2be8041050 2024-05-08T15:47:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7fa161282640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.7:6791/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fa15c139280 con 0x7fa15c0a49a0 2024-05-08T15:47:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7f84d8f5a640 10 monclient: get_auth_request con 0x7f84b4001630 auth_method 0 2024-05-08T15:47:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7f84d3fff640 10 monclient: get_auth_request con 0x7f84c0039c80 auth_method 0 2024-05-08T15:47:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa1440052e0 con 0x7fa15c0a49a0 2024-05-08T15:47:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.377+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fa144046f00 con 0x7fa15c0a49a0 2024-05-08T15:47:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.378+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.81:6802/1326537731 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa13c0442f0 con 0x7fa13c040680 2024-05-08T15:47:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.378+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_get_version_reply(handle=1 version=18) ==== 24+0+0 (unknown 77724187 0 0) 0x7fa144047370 con 0x7fa15c0a49a0 2024-05-08T15:47:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.378+0000 7fa15a7fc640 10 monclient: get_auth_request con 0x7fa13c039b40 auth_method 0 2024-05-08T15:47:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.378+0000 7fa1537fe640 10 monclient: handle_get_version_reply finishing 1 version 18 2024-05-08T15:47:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.378+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== osd.1 v1:172.21.15.7:6800/601836183 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f84b40053a0 con 0x7f84b4001630 2024-05-08T15:47:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.378+0000 7fa15b7fe640 10 monclient: get_auth_request con 0x7fa13c040680 auth_method 0 2024-05-08T15:47:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.379+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== osd.3 v1:172.21.15.81:6802/1326537731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fa13c0442f0 con 0x7fa13c040680 2024-05-08T15:47:12.488 INFO:teuthology.orchestra.run.smithi007.stdout:47244640260 2024-05-08T15:47:12.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T15:47:12.489 INFO:teuthology.orchestra.run.smithi007.stdout:47244640261 2024-05-08T15:47:12.490 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T15:47:12.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.385+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 --> v1:172.21.15.7:6804/3231689356 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32001367b0 con 0x7f31e4040f10 2024-05-08T15:47:12.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.385+0000 7f31f67fc640 1 -- 172.21.15.7:0/439275249 <== osd.0 v1:172.21.15.7:6804/3231689356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3685469309) 0x7f32001367b0 con 0x7f31e4040f10 2024-05-08T15:47:12.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 >> v1:172.21.15.7:6804/3231689356 conn(0x7f31e4040f10 legacy=0x7f31e4043330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 >> v1:172.21.15.81:6812/847627118 conn(0x7f31e403a2c0 legacy=0x7f31e403c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 10 monclient: shutdown 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 >> v1:172.21.15.7:6792/0 conn(0x7f320013e310 legacy=0x7f32001d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f32067b1640 1 -- 172.21.15.7:0/439275249 reap_dead start 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 shutdown_connections 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 >> 172.21.15.7:0/439275249 conn(0x7f32000a9fa0 msgr2=0x7f32000a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 shutdown_connections 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.386+0000 7f3208214640 1 -- 172.21.15.7:0/439275249 wait complete. 2024-05-08T15:47:12.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.393+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 --> v1:172.21.15.7:6800/601836183 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f84b4007030 con 0x7f84b4001630 2024-05-08T15:47:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84d0ff9640 1 -- 172.21.15.7:0/721929879 <== osd.1 v1:172.21.15.7:6800/601836183 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 654699149) 0x7f84b4007030 con 0x7f84b4001630 2024-05-08T15:47:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 --> v1:172.21.15.81:6808/193838907 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2c04075820 con 0x7f2be8041050 2024-05-08T15:47:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f2bfa7fc640 1 -- 172.21.15.7:0/3469258019 <== osd.4 v1:172.21.15.81:6808/193838907 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3580782990) 0x7f2c04075820 con 0x7f2be8041050 2024-05-08T15:47:12.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 >> v1:172.21.15.7:6800/601836183 conn(0x7f84b4001630 legacy=0x7f84b4003af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 >> v1:172.21.15.81:6812/847627118 conn(0x7f84c0039c80 legacy=0x7f84c003c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84ba7fc640 10 monclient: shutdown 2024-05-08T15:47:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84ba7fc640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 >> v1:172.21.15.7:6789/0 conn(0x7f84d40a2700 legacy=0x7f84d41d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84d8f5a640 1 -- 172.21.15.7:0/721929879 reap_dead start 2024-05-08T15:47:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 shutdown_connections 2024-05-08T15:47:12.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 >> 172.21.15.7:0/721929879 conn(0x7f84d40a7320 msgr2=0x7f84d40a5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.394+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 >> v1:172.21.15.81:6808/193838907 conn(0x7f2be8041050 legacy=0x7f2be80434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 >> v1:172.21.15.81:6812/847627118 conn(0x7f2be8039c30 legacy=0x7f2be803c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 10 monclient: shutdown 2024-05-08T15:47:12.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 >> v1:172.21.15.7:6790/0 conn(0x7f2c040a2e10 legacy=0x7f2c041c4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0a178640 1 -- 172.21.15.7:0/3469258019 reap_dead start 2024-05-08T15:47:12.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 shutdown_connections 2024-05-08T15:47:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 shutdown_connections 2024-05-08T15:47:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 >> 172.21.15.7:0/3469258019 conn(0x7f2c040a7320 msgr2=0x7f2c040a5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f84ba7fc640 1 -- 172.21.15.7:0/721929879 wait complete. 2024-05-08T15:47:12.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 shutdown_connections 2024-05-08T15:47:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.395+0000 7f2c0bbdb640 1 -- 172.21.15.7:0/3469258019 wait complete. 2024-05-08T15:47:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.396+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 --> v1:172.21.15.81:6802/1326537731 -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa15c075820 con 0x7fa13c040680 2024-05-08T15:47:12.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa1537fe640 1 -- 172.21.15.7:0/828734723 <== osd.3 v1:172.21.15.81:6802/1326537731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (unknown 0 0 3325991546) 0x7fa15c075820 con 0x7fa13c040680 2024-05-08T15:47:12.499 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-05-08T15:47:12.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T15:47:12.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 >> v1:172.21.15.81:6802/1326537731 conn(0x7fa13c040680 legacy=0x7fa13c042aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 >> v1:172.21.15.81:6812/847627118 conn(0x7fa13c039b40 legacy=0x7fa13c03c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 10 monclient: shutdown 2024-05-08T15:47:12.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 >> v1:172.21.15.7:6791/0 conn(0x7fa15c0a49a0 legacy=0x7fa15c13ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa15b7fe640 1 -- 172.21.15.7:0/828734723 reap_dead start 2024-05-08T15:47:12.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 shutdown_connections 2024-05-08T15:47:12.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 >> 172.21.15.7:0/828734723 conn(0x7fa15c0a9fa0 msgr2=0x7fa15c130ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 shutdown_connections 2024-05-08T15:47:12.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.397+0000 7fa161282640 1 -- 172.21.15.7:0/828734723 wait complete. 2024-05-08T15:47:12.501 INFO:teuthology.orchestra.run.smithi007.stdout:47244640261 2024-05-08T15:47:12.502 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T15:47:12.502 INFO:teuthology.orchestra.run.smithi007.stdout:47244640260 2024-05-08T15:47:12.503 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T15:47:12.503 INFO:teuthology.orchestra.run.smithi007.stdout:47244640262 2024-05-08T15:47:12.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T15:47:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.651+0000 7f3ea6cb5640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.651+0000 7f114144c640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.651+0000 7f114144c640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.651+0000 7f114144c640 10 monclient: monmap: 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.652851+0000 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.652851+0000 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.649 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.651+0000 7f3ea6cb5640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.651+0000 7f3ea6cb5640 10 monclient: monmap: 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.652710+0000 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.652710+0000 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.650 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.652+0000 7f114144c640 1 Processor -- start 2024-05-08T15:47:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.652+0000 7f114144c640 1 -- start start 2024-05-08T15:47:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.652+0000 7f114144c640 10 monclient: init 2024-05-08T15:47:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.652+0000 7f3ea6cb5640 1 Processor -- start 2024-05-08T15:47:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.652+0000 7f3ea6cb5640 1 -- start start 2024-05-08T15:47:12.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.652+0000 7f3ea6cb5640 10 monclient: init 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 10 monclient: _add_conns ranks=[6,2,1,3,7,0,4,8,5] 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 10 monclient(hunting): picked mon.noname-d con 0x7f113c134af0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 10 monclient(hunting): picked mon.noname-b con 0x7f113c0a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 10 monclient(hunting): picked mon.noname-f con 0x7f113c13e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c0a6770 con 0x7f113c13e310 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c0a1660 con 0x7f113c0a49a0 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c0a2910 con 0x7f113c134af0 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f114144c640 10 monclient(hunting): authenticate will time out at 840.767940s 2024-05-08T15:47:12.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f113affd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f113c134af0 0x7f113c134ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60350/0 (socket says 172.21.15.7:60350) 2024-05-08T15:47:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f113affd640 1 -- 172.21.15.7:0/1206271336 learned_addr learned my addr 172.21.15.7:0/1206271336 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f113a7fc640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f113c0a49a0 0x7f113c13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44188/0 (socket says 172.21.15.7:44188) 2024-05-08T15:47:12.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 10 monclient: _add_conns ranks=[3,5,1,7,4,8,6,0,2] 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 10 monclient(hunting): picked mon.noname-g con 0x7f3ea00a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 10 monclient(hunting): picked mon.noname-h con 0x7f3ea00a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 10 monclient(hunting): picked mon.noname-f con 0x7f3ea013e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea00a1660 con 0x7f3ea013e310 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2424779400 0 0) 0x7f113c0a1660 con 0x7f113c0a49a0 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea00a28f0 con 0x7f3ea00a53a0 2024-05-08T15:47:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): my global_id is 74279 2024-05-08T15:47:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1124003520 con 0x7f113c0a49a0 2024-05-08T15:47:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea00a2bd0 con 0x7f3ea00a5780 2024-05-08T15:47:12.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3597034827 0 0) 0x7f113c0a2910 con 0x7f113c134af0 2024-05-08T15:47:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f3ea6cb5640 10 monclient(hunting): authenticate will time out at 840.768385s 2024-05-08T15:47:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3ea4a51640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f3ea00a53a0 0x7f3ea00a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50604/0 (socket says 172.21.15.7:50604) 2024-05-08T15:47:12.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3ea4a51640 1 -- 172.21.15.7:0/2979861657 learned_addr learned my addr 172.21.15.7:0/2979861657 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 10 monclient(hunting): my global_id is 74310 2024-05-08T15:47:12.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.653+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f113c0a1660 con 0x7f113c134af0 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1885964038 0 0) 0x7f113c0a6770 con 0x7f113c13e310 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient(hunting): my global_id is 4267 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f113c0a2910 con 0x7f113c13e310 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2762120695 0 0) 0x7f1124003520 con 0x7f113c0a49a0 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 >> v1:172.21.15.7:6792/0 conn(0x7f113c134af0 legacy=0x7f113c134ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 >> v1:172.21.15.81:6789/0 conn(0x7f113c13e310 legacy=0x7f113c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 monclient: found mon.noname-b 2024-05-08T15:47:12.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c0a2bf0 con 0x7f113c0a49a0 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f11300017b0 con 0x7f113c0a49a0 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1680369111 0 0) 0x7f3ea00a28f0 con 0x7f3ea00a53a0 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: dump: 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.663 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f114144c640 5 monclient: authenticate success, global_id 74279 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f114144c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1130001a10 con 0x7f113c0a49a0 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 1 -- 172.21.15.7:0/1206271336 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1130001c00 con 0x7f113c0a49a0 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f114144c640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f114144c640 4 set_mon_vals no callback set 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f1139ffb640 10 monclient: dump: 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.665 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.666 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): my global_id is 74271 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e90003520 con 0x7f3ea00a53a0 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3323343345 0 0) 0x7f3ea00a1660 con 0x7f3ea013e310 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): my global_id is 4276 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ea00a28f0 con 0x7f3ea013e310 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3743586966 0 0) 0x7f3ea00a2bd0 con 0x7f3ea00a5780 2024-05-08T15:47:12.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient(hunting): my global_id is 4316 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ea00a1660 con 0x7f3ea00a5780 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3954206583 0 0) 0x7f3e90003520 con 0x7f3ea00a53a0 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 >> v1:172.21.15.81:6791/0 conn(0x7f3ea00a5780 legacy=0x7f3ea013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 >> v1:172.21.15.81:6789/0 conn(0x7f3ea013e310 legacy=0x7f3ea0140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.654+0000 7f3e9f7fe640 1 monclient: found mon.noname-g 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea00a6730 con 0x7f3ea00a53a0 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3e940017b0 con 0x7f3ea00a53a0 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: dump: 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.670 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3ea6cb5640 5 monclient: authenticate success, global_id 74271 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3ea6cb5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3e94001a10 con 0x7f3ea00a53a0 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 1 -- 172.21.15.7:0/2979861657 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3e94004be0 con 0x7f3ea00a53a0 2024-05-08T15:47:12.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f114144c640 10 monclient: shutdown 2024-05-08T15:47:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f114144c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f114144c640 1 -- 172.21.15.7:0/1206271336 >> v1:172.21.15.7:6790/0 conn(0x7f113c0a49a0 legacy=0x7f113c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3e9f7fe640 10 monclient: dump: 2024-05-08T15:47:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.672 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3ea6cb5640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f114144c640 1 -- 172.21.15.7:0/1206271336 shutdown_connections 2024-05-08T15:47:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f114144c640 1 -- 172.21.15.7:0/1206271336 >> 172.21.15.7:0/1206271336 conn(0x7f113c0a9fa0 msgr2=0x7f113c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f3ea6cb5640 4 set_mon_vals no callback set 2024-05-08T15:47:12.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f114144c640 1 -- 172.21.15.7:0/1206271336 shutdown_connections 2024-05-08T15:47:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.655+0000 7f114144c640 1 -- 172.21.15.7:0/1206271336 wait complete. 2024-05-08T15:47:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f3ea6cb5640 10 monclient: shutdown 2024-05-08T15:47:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f3ea6cb5640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2979861657 >> v1:172.21.15.81:6790/0 conn(0x7f3ea00a53a0 legacy=0x7f3ea00a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f114144c640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f114144c640 10 monclient: monmap: 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.676 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2979861657 shutdown_connections 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2979861657 >> 172.21.15.7:0/2979861657 conn(0x7f3ea00a9fa0 msgr2=0x7f3ea00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2979861657 shutdown_connections 2024-05-08T15:47:12.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2979861657 wait complete. 2024-05-08T15:47:12.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f114144c640 1 Processor -- start 2024-05-08T15:47:12.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f114144c640 1 -- start start 2024-05-08T15:47:12.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.656+0000 7f114144c640 10 monclient: init 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f3ea6cb5640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f3ea6cb5640 10 monclient: monmap: 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.679 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 10 monclient: _add_conns ranks=[7,8,2,4,0,1,5,6,3] 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 10 monclient(hunting): picked mon.noname-i con 0x7f113c0a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 10 monclient(hunting): picked mon.noname-e con 0x7f113c134af0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 10 monclient(hunting): picked mon.noname-b con 0x7f113c13e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c09e360 con 0x7f113c13e310 2024-05-08T15:47:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f3ea6cb5640 1 Processor -- start 2024-05-08T15:47:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f3ea6cb5640 1 -- start start 2024-05-08T15:47:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c09ce40 con 0x7f113c0a49a0 2024-05-08T15:47:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f113c09e080 con 0x7f113c134af0 2024-05-08T15:47:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f114144c640 10 monclient(hunting): authenticate will time out at 840.772226s 2024-05-08T15:47:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f3ea6cb5640 10 monclient: init 2024-05-08T15:47:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f113a7fc640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f113c134af0 0x7f113c1d1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54518/0 (socket says 172.21.15.7:54518) 2024-05-08T15:47:12.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f113b7fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f113c13e310 0x7f113c1d50c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44190/0 (socket says 172.21.15.7:44190) 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.657+0000 7f113a7fc640 1 -- 172.21.15.7:0/4159375105 learned_addr learned my addr 172.21.15.7:0/4159375105 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953612623 0 0) 0x7f113c09ce40 con 0x7f113c0a49a0 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): my global_id is 4282 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1128003520 con 0x7f113c0a49a0 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3505787760 0 0) 0x7f113c09e080 con 0x7f113c134af0 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): my global_id is 4283 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f113c09ce40 con 0x7f113c134af0 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032562312 0 0) 0x7f113c09e360 con 0x7f113c13e310 2024-05-08T15:47:12.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): my global_id is 74288 2024-05-08T15:47:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f113c09e080 con 0x7f113c13e310 2024-05-08T15:47:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 10 monclient: _add_conns ranks=[6,0,7,2,4,1,8,5,3] 2024-05-08T15:47:12.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 10 monclient(hunting): picked mon.noname-d con 0x7f3ea00a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 533635394 0 0) 0x7f113c09e080 con 0x7f113c13e310 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f113c09e360 con 0x7f113c13e310 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2701841879 0 0) 0x7f1128003520 con 0x7f113c0a49a0 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f113c09e080 con 0x7f113c0a49a0 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f112c0014f0 con 0x7f113c0a49a0 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): dump: 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.685 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.686 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1134001cb0 con 0x7f113c13e310 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): dump: 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.687 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3157516304 0 0) 0x7f113c09ce40 con 0x7f113c134af0 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1128003520 con 0x7f113c134af0 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1130001790 con 0x7f113c134af0 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 10 monclient(hunting): picked mon.noname-a con 0x7f3ea00a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient(hunting): dump: 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.689 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.690 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.690 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.690 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.690 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 10 monclient(hunting): picked mon.noname-i con 0x7f3ea013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea00a43c0 con 0x7f3ea00a5780 2024-05-08T15:47:12.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea0132e40 con 0x7f3ea00a53a0 2024-05-08T15:47:12.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea0134890 con 0x7f3ea013e310 2024-05-08T15:47:12.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2348878013 0 0) 0x7f113c09e360 con 0x7f113c13e310 2024-05-08T15:47:12.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f3ea6cb5640 10 monclient(hunting): authenticate will time out at 840.773320s 2024-05-08T15:47:12.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9ffff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3ea00a5780 0x7f3ea01d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57350/0 (socket says 172.21.15.7:57350) 2024-05-08T15:47:12.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9ffff640 1 -- 172.21.15.7:0/2506746124 learned_addr learned my addr 172.21.15.7:0/2506746124 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 >> v1:172.21.15.7:6793/0 conn(0x7f113c134af0 legacy=0x7f113c1d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 >> v1:172.21.15.81:6792/0 conn(0x7f113c0a49a0 legacy=0x7f113c09fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 monclient: found mon.b 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c09c9c0 con 0x7f113c13e310 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.658+0000 7f11237fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f114144c640 5 monclient: authenticate success, global_id 74288 2024-05-08T15:47:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645561616 0 0) 0x7f3ea0134890 con 0x7f3ea013e310 2024-05-08T15:47:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): my global_id is 4291 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e84003520 con 0x7f3ea013e310 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095635203 0 0) 0x7f3ea0132e40 con 0x7f3ea00a53a0 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): my global_id is 74319 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ea0134890 con 0x7f3ea00a53a0 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1594059963 0 0) 0x7f3ea00a1660 con 0x7f3ea00a5780 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 10 monclient(hunting): my global_id is 4356 2024-05-08T15:47:12.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ea0132e40 con 0x7f3ea00a5780 2024-05-08T15:47:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f114144c640 10 monclient: _renew_subs 2024-05-08T15:47:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f114144c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f113c0a0280 con 0x7f113c13e310 2024-05-08T15:47:12.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f114144c640 10 monclient: _renew_subs 2024-05-08T15:47:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f114144c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.659+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f113c1cc910 con 0x7f113c13e310 2024-05-08T15:47:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 324445011 0 0) 0x7f3e84003520 con 0x7f3ea013e310 2024-05-08T15:47:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3ea00a1660 con 0x7f3ea013e310 2024-05-08T15:47:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3e98001c20 con 0x7f3ea013e310 2024-05-08T15:47:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:47:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.698 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.699 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3728443128 0 0) 0x7f3ea0134890 con 0x7f3ea00a53a0 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3e84003520 con 0x7f3ea00a53a0 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 928232688 0 0) 0x7f3ea0132e40 con 0x7f3ea00a5780 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3ea0134890 con 0x7f3ea00a5780 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3e94004b20 con 0x7f3ea00a53a0 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.700 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3e8c001990 con 0x7f3ea00a5780 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:12.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.702 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1061985469 0 0) 0x7f3ea00a1660 con 0x7f3ea013e310 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 >> v1:172.21.15.7:6792/0 conn(0x7f3ea00a53a0 legacy=0x7f3ea012e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 >> v1:172.21.15.7:6789/0 conn(0x7f3ea00a5780 legacy=0x7f3ea01d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 monclient: found mon.i 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01339a0 con 0x7f3ea013e310 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3ea6cb5640 5 monclient: authenticate success, global_id 4291 2024-05-08T15:47:12.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3ea6cb5640 10 monclient: _renew_subs 2024-05-08T15:47:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3ea6cb5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea01386d0 con 0x7f3ea013e310 2024-05-08T15:47:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3ea6cb5640 10 monclient: _renew_subs 2024-05-08T15:47:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3ea6cb5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3e98003990 con 0x7f3ea013e310 2024-05-08T15:47:12.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.660+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f3ea013cfa0 con 0x7f3ea013e310 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1134002010 con 0x7f113c13e310 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11237fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1134001e50 con 0x7f113c13e310 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11237fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11237fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11237fe640 10 monclient: dump: 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.705 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11397fa640 4 set_mon_vals no callback set 2024-05-08T15:47:12.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f3e9d7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3e98003e80 con 0x7f3ea013e310 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f3e9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f3e9d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f3e9d7fa640 10 monclient: dump: 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.708 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f3e9effd640 4 set_mon_vals no callback set 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f3ea6cb5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f3ea6cb5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f3ea6cb5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e80005080 con 0x7f3ea013e310 2024-05-08T15:47:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.661+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f1134003f60 con 0x7f113c13e310 2024-05-08T15:47:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f114144c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f3e9800f200 con 0x7f3ea013e310 2024-05-08T15:47:12.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f114144c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f114144c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.662+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c005080 con 0x7f113c13e310 2024-05-08T15:47:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.663+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f3e98045c40 con 0x7f3ea013e310 2024-05-08T15:47:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.663+0000 7f3ea4a51640 10 monclient: get_auth_request con 0x7f3e8403a390 auth_method 0 2024-05-08T15:47:12.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.663+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f1134046e70 con 0x7f113c13e310 2024-05-08T15:47:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.663+0000 7f113affd640 10 monclient: get_auth_request con 0x7f112803a5b0 auth_method 0 2024-05-08T15:47:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.668+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f3e98014dd0 con 0x7f3ea013e310 2024-05-08T15:47:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.668+0000 7f3e9d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.668+0000 7f3e9d7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.672+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f1134016020 con 0x7f113c13e310 2024-05-08T15:47:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.672+0000 7f11237fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.672+0000 7f11237fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.692+0000 7fe1349b2640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.692+0000 7fe1349b2640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.692+0000 7fe1349b2640 10 monclient: monmap: 2024-05-08T15:47:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.693400+0000 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.693400+0000 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.718 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.693+0000 7fe1349b2640 1 Processor -- start 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.693+0000 7fe1349b2640 1 -- start start 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.693+0000 7fe1349b2640 10 monclient: init 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.693+0000 7fe1349b2640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.693+0000 7fe1349b2640 10 monclient: _add_conns ranks=[2,4,7,8,0,5,1,3,6] 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.693+0000 7fe1349b2640 10 monclient(hunting): picked mon.noname-b con 0x7fe1300a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe1349b2640 10 monclient(hunting): picked mon.noname-c con 0x7fe1300a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe1349b2640 10 monclient(hunting): picked mon.noname-i con 0x7fe13013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe1349b2640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1300a1660 con 0x7fe1300a53a0 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe1349b2640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1300a28f0 con 0x7fe1300a5780 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12dd9b640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fe1300a5780 0x7fe13013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51044/0 (socket says 172.21.15.7:51044) 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12dd9b640 1 -- 172.21.15.7:0/1218176831 learned_addr learned my addr 172.21.15.7:0/1218176831 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe1349b2640 1 -- 172.21.15.7:0/1218176831 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1300a2bd0 con 0x7fe13013e310 2024-05-08T15:47:12.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe1349b2640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe1349b2640 10 monclient(hunting): authenticate will time out at 840.808899s 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089749849 0 0) 0x7fe1300a1660 con 0x7fe1300a53a0 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): my global_id is 74297 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe118003520 con 0x7fe1300a53a0 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1733524989 0 0) 0x7fe1300a2bd0 con 0x7fe13013e310 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 10 monclient(hunting): my global_id is 4300 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.694+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1300a1660 con 0x7fe13013e310 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 743738308 0 0) 0x7fe1300a28f0 con 0x7fe1300a5780 2024-05-08T15:47:12.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient(hunting): my global_id is 4288 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1300a2bd0 con 0x7fe1300a5780 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 599420519 0 0) 0x7fe118003520 con 0x7fe1300a53a0 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 >> v1:172.21.15.81:6792/0 conn(0x7fe13013e310 legacy=0x7fe130140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 >> v1:172.21.15.7:6791/0 conn(0x7fe1300a5780 legacy=0x7fe13013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 monclient: found mon.noname-b 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1300a6730 con 0x7fe1300a53a0 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe1200017b0 con 0x7fe1300a53a0 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe1349b2640 5 monclient: authenticate success, global_id 74297 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe1349b2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: dump: 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.722 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe1349b2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe120001a80 con 0x7fe1300a53a0 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 1 -- 172.21.15.7:0/1218176831 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe120003950 con 0x7fe1300a53a0 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe1349b2640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe1349b2640 4 set_mon_vals no callback set 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.695+0000 7fe12d59a640 10 monclient: dump: 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.696+0000 7fe1349b2640 10 monclient: shutdown 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.696+0000 7fe1349b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.696+0000 7fe1349b2640 1 -- 172.21.15.7:0/1218176831 >> v1:172.21.15.7:6790/0 conn(0x7fe1300a53a0 legacy=0x7fe1300a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.696+0000 7fe1349b2640 1 -- 172.21.15.7:0/1218176831 shutdown_connections 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.696+0000 7fe1349b2640 1 -- 172.21.15.7:0/1218176831 >> 172.21.15.7:0/1218176831 conn(0x7fe1300a9fa0 msgr2=0x7fe1300aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.696+0000 7fe1349b2640 1 -- 172.21.15.7:0/1218176831 shutdown_connections 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.696+0000 7fe1349b2640 1 -- 172.21.15.7:0/1218176831 wait complete. 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.697+0000 7effaa8df640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.697+0000 7effaa8df640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.697+0000 7fe1349b2640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.697+0000 7effaa8df640 10 monclient: monmap: 2024-05-08T15:47:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.698660+0000 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.698660+0000 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.727 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.697+0000 7fe1349b2640 10 monclient: monmap: 2024-05-08T15:47:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.728 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 1 Processor -- start 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 1 -- start start 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient: init 2024-05-08T15:47:12.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7effaa8df640 1 Processor -- start 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient: _add_conns ranks=[3,8,5,6,7,4,2,1,0] 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient(hunting): picked mon.noname-g con 0x7fe1300a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient(hunting): picked mon.noname-e con 0x7fe1300a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient(hunting): picked mon.noname-h con 0x7fe13013e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1301346b0 con 0x7fe1300a53a0 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1300a43c0 con 0x7fe13013e310 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe130136100 con 0x7fe1300a5780 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.698+0000 7fe1349b2640 10 monclient(hunting): authenticate will time out at 840.813592s 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe12dd9b640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fe1300a5780 0x7fe1301d1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54530/0 (socket says 172.21.15.7:54530) 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe12dd9b640 1 -- 172.21.15.7:0/643826400 learned_addr learned my addr 172.21.15.7:0/643826400 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3415037568 0 0) 0x7fe130136100 con 0x7fe1300a5780 2024-05-08T15:47:12.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 10 monclient(hunting): my global_id is 4292 2024-05-08T15:47:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe110003520 con 0x7fe1300a5780 2024-05-08T15:47:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 1 -- start start 2024-05-08T15:47:12.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient: init 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient: _add_conns ranks=[0,1,7,5,4,3,6,8,2] 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient(hunting): picked mon.noname-a con 0x7effa40a2700 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient(hunting): picked mon.noname-f con 0x7effa4072710 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient(hunting): picked mon.noname-i con 0x7effa40a31f0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa409fd50 con 0x7effa40a2700 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa40a0030 con 0x7effa4072710 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa40a4230 con 0x7effa40a31f0 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4139073782 0 0) 0x7fe110003520 con 0x7fe1300a5780 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7effaa8df640 10 monclient(hunting): authenticate will time out at 840.814486s 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe130136100 con 0x7fe1300a5780 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe124001b30 con 0x7fe1300a5780 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.699+0000 7fe11effd640 10 monclient(hunting): dump: 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.734 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa3fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7effa40a2700 0x7effa40a2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57356/0 (socket says 172.21.15.7:57356) 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa3fff640 1 -- 172.21.15.7:0/2903401094 learned_addr learned my addr 172.21.15.7:0/2903401094 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3481105329 0 0) 0x7fe1301346b0 con 0x7fe1300a53a0 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient(hunting): my global_id is 74280 2024-05-08T15:47:12.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe110003520 con 0x7fe1300a53a0 2024-05-08T15:47:12.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825165007 0 0) 0x7effa409fd50 con 0x7effa40a2700 2024-05-08T15:47:12.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3871949368 0 0) 0x7fe130136100 con 0x7fe1300a5780 2024-05-08T15:47:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): my global_id is 4365 2024-05-08T15:47:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff90003520 con 0x7effa40a2700 2024-05-08T15:47:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7f3dfc74b640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 862751233 0 0) 0x7effa40a4230 con 0x7effa40a31f0 2024-05-08T15:47:12.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 >> v1:172.21.15.81:6791/0 conn(0x7fe13013e310 legacy=0x7fe1301d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 >> v1:172.21.15.81:6790/0 conn(0x7fe1300a53a0 legacy=0x7fe130137ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 1 monclient: found mon.e 2024-05-08T15:47:12.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): my global_id is 4309 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa409fd50 con 0x7effa40a31f0 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1301385e0 con 0x7fe1300a5780 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe11effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe1349b2640 5 monclient: authenticate success, global_id 4292 2024-05-08T15:47:12.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465880142 0 0) 0x7effa40a0030 con 0x7effa4072710 2024-05-08T15:47:12.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe1349b2640 10 monclient: _renew_subs 2024-05-08T15:47:12.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe1349b2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe1301c9730 con 0x7fe1300a5780 2024-05-08T15:47:12.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe1349b2640 10 monclient: _renew_subs 2024-05-08T15:47:12.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe1349b2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fe1301353f0 con 0x7fe1300a5780 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 10 monclient(hunting): my global_id is 4285 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa40a4230 con 0x7effa4072710 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 139688217 0 0) 0x7eff90003520 con 0x7effa40a2700 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 >> v1:172.21.15.81:6792/0 conn(0x7effa40a31f0 legacy=0x7effa413d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 >> v1:172.21.15.81:6789/0 conn(0x7effa4072710 legacy=0x7effa4139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 1 monclient: found mon.noname-a 2024-05-08T15:47:12.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa409e9f0 con 0x7effa40a2700 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effaa8df640 5 monclient: authenticate success, global_id 4365 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effaa8df640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7eff940017b0 con 0x7effa40a2700 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: dump: 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.745 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effaa8df640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7f3dfc74b640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.700+0000 7f3dfc74b640 10 monclient: monmap: 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.701633+0000 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.701633+0000 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.747 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7eff94001a10 con 0x7effa40a2700 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 1 -- 172.21.15.7:0/2903401094 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7eff94004be0 con 0x7effa40a2700 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effa37fe640 10 monclient: dump: 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.749 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe124004a90 con 0x7fe1300a5780 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7fe11effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe124005000 con 0x7fe1300a5780 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7fe11effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7fe11effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7fe11effd640 10 monclient: dump: 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.750 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7fe12cd99640 4 set_mon_vals no callback set 2024-05-08T15:47:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effaa8df640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7effaa8df640 4 set_mon_vals no callback set 2024-05-08T15:47:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fe1240051a0 con 0x7fe1300a5780 2024-05-08T15:47:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7fe1349b2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.701+0000 7f3dfc74b640 1 Processor -- start 2024-05-08T15:47:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7f3dfc74b640 1 -- start start 2024-05-08T15:47:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7f3dfc74b640 10 monclient: init 2024-05-08T15:47:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7fe1349b2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7fe1349b2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe114005080 con 0x7fe1300a5780 2024-05-08T15:47:12.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7effaa8df640 10 monclient: shutdown 2024-05-08T15:47:12.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7effaa8df640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7effaa8df640 1 -- 172.21.15.7:0/2903401094 >> v1:172.21.15.7:6789/0 conn(0x7effa40a2700 legacy=0x7effa40a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7f3dfc74b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7f3dfc74b640 10 monclient: _add_conns ranks=[4,6,7,5,1,2,0,3,8] 2024-05-08T15:47:12.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.702+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fe124046f00 con 0x7fe1300a5780 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7effaa8df640 1 -- 172.21.15.7:0/2903401094 shutdown_connections 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7effaa8df640 1 -- 172.21.15.7:0/2903401094 >> 172.21.15.7:0/2903401094 conn(0x7effa40a7320 msgr2=0x7effa40a37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7effaa8df640 1 -- 172.21.15.7:0/2903401094 shutdown_connections 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 10 monclient(hunting): picked mon.noname-c con 0x7f3df40724d0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 10 monclient(hunting): picked mon.noname-d con 0x7f3df40728b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 10 monclient(hunting): picked mon.noname-i con 0x7f3df4072d30 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df40a67b0 con 0x7f3df40724d0 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df40a2670 con 0x7f3df40728b0 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df40a3920 con 0x7f3df4072d30 2024-05-08T15:47:12.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfc74b640 10 monclient(hunting): authenticate will time out at 840.817871s 2024-05-08T15:47:12.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfa4e7640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f3df40724d0 0x7f3df406ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51048/0 (socket says 172.21.15.7:51048) 2024-05-08T15:47:12.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7f3dfa4e7640 1 -- 172.21.15.7:0/562696942 learned_addr learned my addr 172.21.15.7:0/562696942 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7fe12e59c640 10 monclient: get_auth_request con 0x7fe1100399f0 auth_method 0 2024-05-08T15:47:12.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.703+0000 7effaa8df640 1 -- 172.21.15.7:0/2903401094 wait complete. 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1444506108 0 0) 0x7f3df40a67b0 con 0x7f3df40724d0 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): my global_id is 4297 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3de0003520 con 0x7f3df40724d0 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616758719 0 0) 0x7f3df40a2670 con 0x7f3df40728b0 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): my global_id is 74328 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3df40a67b0 con 0x7f3df40728b0 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2587898833 0 0) 0x7f3df40a3920 con 0x7f3df4072d30 2024-05-08T15:47:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient(hunting): my global_id is 4318 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3df40a2670 con 0x7f3df4072d30 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7effaa8df640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7effaa8df640 10 monclient: monmap: 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7effaa8df640 1 Processor -- start 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7effaa8df640 1 -- start start 2024-05-08T15:47:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7effaa8df640 10 monclient: init 2024-05-08T15:47:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1313450015 0 0) 0x7f3de0003520 con 0x7f3df40724d0 2024-05-08T15:47:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 >> v1:172.21.15.81:6792/0 conn(0x7f3df4072d30 legacy=0x7f3df41448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 >> v1:172.21.15.7:6792/0 conn(0x7f3df40728b0 legacy=0x7f3df4141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 monclient: found mon.noname-c 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df40a3c00 con 0x7f3df40724d0 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3de40017d0 con 0x7f3df40724d0 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: dump: 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3dfc74b640 5 monclient: authenticate success, global_id 4297 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3dfc74b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3de4001a30 con 0x7f3df40724d0 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 1 -- 172.21.15.7:0/562696942 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3de4001be0 con 0x7f3df40724d0 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3df94e5640 10 monclient: dump: 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3dfc74b640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.704+0000 7f3dfc74b640 4 set_mon_vals no callback set 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 10 monclient: _add_conns ranks=[3,5,1,4,7,6,2,8,0] 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 10 monclient(hunting): picked mon.noname-g con 0x7effa4072710 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 10 monclient(hunting): picked mon.noname-h con 0x7effa40a2700 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 10 monclient(hunting): picked mon.noname-f con 0x7effa40a31f0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa4132880 con 0x7effa40a31f0 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa41318b0 con 0x7effa4072710 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effa4072b60 con 0x7effa40a2700 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effaa8df640 10 monclient(hunting): authenticate will time out at 840.820042s 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7fde72c0f640 10 monclient: get_monmap_and_config 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7fde72c0f640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7fde72c0f640 10 monclient: monmap: 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:12.706436+0000 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:12.706436+0000 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.766 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.767 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa3fff640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7effa4072710 0x7effa413dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50626/0 (socket says 172.21.15.7:50626) 2024-05-08T15:47:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa3fff640 1 -- 172.21.15.7:0/3301173576 learned_addr learned my addr 172.21.15.7:0/3301173576 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7f3dfc74b640 10 monclient: shutdown 2024-05-08T15:47:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7f3dfc74b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7f3dfc74b640 1 -- 172.21.15.7:0/562696942 >> v1:172.21.15.7:6791/0 conn(0x7f3df40724d0 legacy=0x7f3df406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3020091643 0 0) 0x7effa41318b0 con 0x7effa4072710 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): my global_id is 74289 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff84003520 con 0x7effa4072710 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465155104 0 0) 0x7effa4072b60 con 0x7effa40a2700 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 10 monclient(hunting): my global_id is 4334 2024-05-08T15:47:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa41318b0 con 0x7effa40a2700 2024-05-08T15:47:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7f3dfc74b640 1 -- 172.21.15.7:0/562696942 shutdown_connections 2024-05-08T15:47:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.705+0000 7f3dfc74b640 1 -- 172.21.15.7:0/562696942 >> 172.21.15.7:0/562696942 conn(0x7f3df40a6aa0 msgr2=0x7f3df40a6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7f3dfc74b640 1 -- 172.21.15.7:0/562696942 shutdown_connections 2024-05-08T15:47:12.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7f3dfc74b640 1 -- 172.21.15.7:0/562696942 wait complete. 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1189190864 0 0) 0x7effa4132880 con 0x7effa40a31f0 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): my global_id is 4294 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7effa4072b60 con 0x7effa40a31f0 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1810961322 0 0) 0x7eff84003520 con 0x7effa4072710 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7effa4132880 con 0x7effa4072710 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7eff94004b20 con 0x7effa4072710 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.771 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1045361122 0 0) 0x7effa41318b0 con 0x7effa40a2700 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7eff84003520 con 0x7effa40a2700 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7eff8c001950 con 0x7effa40a2700 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.773 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2347872925 0 0) 0x7effa4072b60 con 0x7effa40a31f0 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7effa41318b0 con 0x7effa40a31f0 2024-05-08T15:47:12.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7eff9c001cf0 con 0x7effa40a31f0 2024-05-08T15:47:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7fde72c0f640 1 Processor -- start 2024-05-08T15:47:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7fde72c0f640 1 -- start start 2024-05-08T15:47:12.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7fde72c0f640 10 monclient: init 2024-05-08T15:47:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.776 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3559591388 0 0) 0x7effa4132880 con 0x7effa4072710 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 >> v1:172.21.15.81:6791/0 conn(0x7effa40a2700 legacy=0x7effa41d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 >> v1:172.21.15.81:6789/0 conn(0x7effa40a31f0 legacy=0x7effa41d50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 monclient: found mon.g 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa4131de0 con 0x7effa4072710 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effa17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effaa8df640 5 monclient: authenticate success, global_id 74289 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effaa8df640 10 monclient: _renew_subs 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effaa8df640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7effa41325a0 con 0x7effa4072710 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effaa8df640 10 monclient: _renew_subs 2024-05-08T15:47:12.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effaa8df640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.706+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7effa406b470 con 0x7effa4072710 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7eff94001a10 con 0x7effa4072710 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa17fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7eff94001790 con 0x7effa4072710 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa17fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa17fa640 10 monclient: dump: 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.779 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa2ffd640 4 set_mon_vals no callback set 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7f3dfc74b640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7f3dfc74b640 10 monclient: monmap: 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.781 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.782 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7fde72c0f640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7fde72c0f640 10 monclient: _add_conns ranks=[8,3,5,4,7,2,1,6,0] 2024-05-08T15:47:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7fde72c0f640 10 monclient(hunting): picked mon.noname-e con 0x7fde6c06e520 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:12.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7fde709ab640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fde6c06e520 0x7fde6c06e900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54538/0 (socket says 172.21.15.7:54538) 2024-05-08T15:47:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7f3dfc74b640 1 Processor -- start 2024-05-08T15:47:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7fde709ab640 1 -- 172.21.15.7:0/924643462 learned_addr learned my addr 172.21.15.7:0/924643462 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.707+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7eff94010400 con 0x7effa4072710 2024-05-08T15:47:12.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7f3dfc74b640 1 -- start start 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7f3dfc74b640 10 monclient: init 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde72c0f640 10 monclient(hunting): picked mon.noname-g con 0x7fde6c06f010 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde72c0f640 10 monclient(hunting): picked mon.noname-h con 0x7fde6c13aa90 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde6c070be0 con 0x7fde6c06f010 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde6c0711a0 con 0x7fde6c13aa90 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde6c06fe50 con 0x7fde6c06e520 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde72c0f640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde72c0f640 10 monclient(hunting): authenticate will time out at 840.822835s 2024-05-08T15:47:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7eff94046d80 con 0x7effa4072710 2024-05-08T15:47:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7eff9bfff640 10 monclient: get_auth_request con 0x7eff8403a580 auth_method 0 2024-05-08T15:47:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7effaa8df640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7effaa8df640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2162182053 0 0) 0x7fde6c070be0 con 0x7fde6c06f010 2024-05-08T15:47:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7effaa8df640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde6b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde6b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff88005080 con 0x7effa4072710 2024-05-08T15:47:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde6b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde6b7fe640 10 monclient(hunting): my global_id is 74298 2024-05-08T15:47:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.708+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde5c003520 con 0x7fde6c06f010 2024-05-08T15:47:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 10 monclient: _add_conns ranks=[3,7,5,0,8,6,4,1,2] 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954914461 0 0) 0x7fde6c0711a0 con 0x7fde6c13aa90 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient(hunting): my global_id is 4343 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde6c070be0 con 0x7fde6c13aa90 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1624012932 0 0) 0x7fde5c003520 con 0x7fde6c06f010 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 >> v1:172.21.15.7:6793/0 conn(0x7fde6c06e520 legacy=0x7fde6c06e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 >> v1:172.21.15.81:6791/0 conn(0x7fde6c13aa90 legacy=0x7fde6c13ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 monclient: found mon.noname-g 2024-05-08T15:47:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c072cc0 con 0x7fde6c06f010 2024-05-08T15:47:12.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 10 monclient(hunting): picked mon.noname-g con 0x7f3df40724d0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fde580017b0 con 0x7fde6c06f010 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde6b7fe640 10 monclient: dump: 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.790 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde72c0f640 5 monclient: authenticate success, global_id 74298 2024-05-08T15:47:12.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 10 monclient(hunting): picked mon.noname-i con 0x7f3df40728b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:12.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7fde72c0f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfa4e7640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f3df40724d0 0x7f3df41457f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50646/0 (socket says 172.21.15.7:50646) 2024-05-08T15:47:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 10 monclient(hunting): picked mon.noname-h con 0x7f3df4072d30 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:12.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df40a2ba0 con 0x7f3df40724d0 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df40732c0 con 0x7f3df4072d30 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df4140510 con 0x7f3df40728b0 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfc74b640 10 monclient(hunting): authenticate will time out at 840.823949s 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3dfa4e7640 1 -- 172.21.15.7:0/3392155336 learned_addr learned my addr 172.21.15.7:0/3392155336 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786182792 0 0) 0x7f3df4140510 con 0x7f3df40728b0 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): my global_id is 4327 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dd8003520 con 0x7f3df40728b0 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4154898665 0 0) 0x7f3df40732c0 con 0x7f3df4072d30 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): my global_id is 4352 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3df4140510 con 0x7f3df4072d30 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 115110326 0 0) 0x7f3dd8003520 con 0x7f3df40728b0 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3df40732c0 con 0x7f3df40728b0 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3df0001510 con 0x7f3df40728b0 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.797 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.709+0000 7f3deaffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1026404203 0 0) 0x7f3df4140510 con 0x7f3df4072d30 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3dd8003520 con 0x7f3df4072d30 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3dec001b60 con 0x7f3df4072d30 2024-05-08T15:47:12.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.799 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3620497560 0 0) 0x7f3df40732c0 con 0x7f3df40728b0 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 >> v1:172.21.15.81:6791/0 conn(0x7f3df4072d30 legacy=0x7f3df41d8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 >> v1:172.21.15.81:6790/0 conn(0x7f3df40724d0 legacy=0x7f3df41457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 monclient: found mon.i 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df40a08d0 con 0x7f3df40728b0 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3dfc74b640 5 monclient: authenticate success, global_id 4327 2024-05-08T15:47:12.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3dfc74b640 10 monclient: _renew_subs 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3dfc74b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3dfc74b640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3df413fc70 con 0x7f3df40728b0 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3dfc74b640 10 monclient: _renew_subs 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3dfc74b640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3dfc74b640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f3df409ff80 con 0x7f3df40728b0 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3df00038a0 con 0x7f3df40728b0 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3df0003d90 con 0x7f3df40728b0 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3deaffd640 10 monclient: dump: 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.801 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.710+0000 7f3df8ce4640 4 set_mon_vals no callback set 2024-05-08T15:47:12.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.711+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f3df000f110 con 0x7f3df40728b0 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.711+0000 7f3de8ff9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.711+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f3df0045bd0 con 0x7f3df40728b0 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.711+0000 7f3de8ff9640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.711+0000 7f3de8ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.711+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df40757f0 con 0x7f3df40728b0 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.711+0000 7f3dfa4e7640 10 monclient: get_auth_request con 0x7f3dd8039f50 auth_method 0 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.712+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fe124016090 con 0x7fe1300a5780 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.712+0000 7fe11effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.712+0000 7fe11effd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7eff94015fd0 con 0x7effa4072710 2024-05-08T15:47:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7effa17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7effa17fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fde58001a10 con 0x7fde6c06f010 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7fde6b7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7fde6b7fe640 1 -- 172.21.15.7:0/924643462 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fde58003950 con 0x7fde6c06f010 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7fde6b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7fde6b7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.714+0000 7fde6b7fe640 10 monclient: dump: 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.805 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.715+0000 7fde72c0f640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:12.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.715+0000 7fde72c0f640 4 set_mon_vals no callback set 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.716+0000 7fde72c0f640 10 monclient: shutdown 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.716+0000 7fde72c0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.716+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 >> v1:172.21.15.81:6790/0 conn(0x7fde6c06f010 legacy=0x7fde6c139370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.716+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 shutdown_connections 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.716+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 >> 172.21.15.7:0/924643462 conn(0x7fde6c0a6280 msgr2=0x7fde6c0a2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.716+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 shutdown_connections 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.716+0000 7fde72c0f640 1 -- 172.21.15.7:0/924643462 wait complete. 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.717+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f3df0014d00 con 0x7f3df40728b0 2024-05-08T15:47:12.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.717+0000 7f3deaffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.717+0000 7f3deaffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.717+0000 7fde72c0f640 10 monclient: build_initial_monmap 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.717+0000 7fde72c0f640 10 monclient: monmap: 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:12.811 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.718+0000 7fde72c0f640 1 Processor -- start 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.718+0000 7fde72c0f640 1 -- start start 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.718+0000 7fde72c0f640 10 monclient: init 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 10 monclient: _add_conns ranks=[4,8,1,5,0,3,7,2,6] 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 10 monclient(hunting): picked mon.noname-c con 0x7fde6c06e520 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:12.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 10 monclient(hunting): picked mon.noname-e con 0x7fde6c06f010 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 10 monclient(hunting): picked mon.noname-f con 0x7fde6c13aa90 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde6c13ef00 con 0x7fde6c13aa90 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde6c140550 con 0x7fde6c06e520 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde6c1cc8b0 con 0x7fde6c06f010 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde72c0f640 10 monclient(hunting): authenticate will time out at 840.833778s 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde709ab640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fde6c06e520 0x7fde6c1ccce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51056/0 (socket says 172.21.15.7:51056) 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde709ab640 1 -- 172.21.15.7:0/1044555450 learned_addr learned my addr 172.21.15.7:0/1044555450 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4206182422 0 0) 0x7fde6c1cc8b0 con 0x7fde6c06f010 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): my global_id is 4301 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde50003520 con 0x7fde6c06f010 2024-05-08T15:47:12.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4066827181 0 0) 0x7fde6c140550 con 0x7fde6c06e520 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 10 monclient(hunting): my global_id is 4306 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.719+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde6c1cc8b0 con 0x7fde6c06e520 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1867710880 0 0) 0x7fde6c1cc8b0 con 0x7fde6c06e520 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fde6c140550 con 0x7fde6c06e520 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fde60001a50 con 0x7fde6c06e520 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.814 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:12.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4208361530 0 0) 0x7fde50003520 con 0x7fde6c06f010 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fde50006bb0 con 0x7fde6c06f010 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fde580016b0 con 0x7fde6c06f010 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient(hunting): dump: 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.816 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 345519094 0 0) 0x7fde50006bb0 con 0x7fde6c06f010 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 >> v1:172.21.15.7:6791/0 conn(0x7fde6c06e520 legacy=0x7fde6c1ccce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 >> v1:172.21.15.81:6789/0 conn(0x7fde6c13aa90 legacy=0x7fde6c1d5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient: _finish_hunting 0 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 monclient: found mon.e 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c0706b0 con 0x7fde6c06f010 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient: _finish_auth 0 2024-05-08T15:47:12.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 10 monclient: _check_auth_tickets 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde697fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde72c0f640 5 monclient: authenticate success, global_id 4301 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde72c0f640 10 monclient: _renew_subs 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde72c0f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c070160 con 0x7fde6c06f010 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde72c0f640 10 monclient: _renew_subs 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde72c0f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.720+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fde6c141210 con 0x7fde6c06f010 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fde58001a10 con 0x7fde6c06f010 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde697fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fde58003a10 con 0x7fde6c06f010 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde697fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde697fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde697fa640 10 monclient: dump: 2024-05-08T15:47:12.818 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:12.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde6affd640 4 set_mon_vals no callback set 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fde5800f170 con 0x7fde6c06f010 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.722+0000 7fde72c0f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.723+0000 7fde72c0f640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.723+0000 7fde72c0f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.723+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4c005080 con 0x7fde6c06f010 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.723+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fde58045a30 con 0x7fde6c06f010 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.723+0000 7fde709ab640 10 monclient: get_auth_request con 0x7fde5003a410 auth_method 0 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.732+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fde58014c20 con 0x7fde6c06f010 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.732+0000 7fde697fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:12.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.732+0000 7fde697fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.878+0000 7f114144c640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-05-08T15:47:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.878+0000 7f114144c640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-05-08T15:47:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.878+0000 7f114144c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.878+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f111c005370 con 0x7f113c13e310 2024-05-08T15:47:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.879+0000 7f11237fe640 1 -- 172.21.15.7:0/4159375105 <== mon.2 v1:172.21.15.7:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 3056272221) 0x7f1134015920 con 0x7f113c13e310 2024-05-08T15:47:12.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.879+0000 7f11237fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-05-08T15:47:12.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.879+0000 7f11237fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:12.876 INFO:teuthology.orchestra.run.smithi007.stdout:55834574852 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 >> v1:172.21.15.81:6812/847627118 conn(0x7f112803a5b0 legacy=0x7f112803ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 10 monclient: shutdown 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 >> v1:172.21.15.7:6790/0 conn(0x7f113c13e310 legacy=0x7f113c1d50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 shutdown_connections 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 >> 172.21.15.7:0/4159375105 conn(0x7f113c0a9fa0 msgr2=0x7f113c130e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 shutdown_connections 2024-05-08T15:47:12.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.881+0000 7f114144c640 1 -- 172.21.15.7:0/4159375105 wait complete. 2024-05-08T15:47:12.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.0 2024-05-08T15:47:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.912+0000 7fe1349b2640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-05-08T15:47:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.912+0000 7fe1349b2640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-05-08T15:47:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.912+0000 7fe1349b2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:12.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.912+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe114005370 con 0x7fe1300a5780 2024-05-08T15:47:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.913+0000 7fe11effd640 1 -- 172.21.15.7:0/643826400 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (unknown 383520633 0 1444633209) 0x7fe124015990 con 0x7fe1300a5780 2024-05-08T15:47:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.913+0000 7fe11effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-05-08T15:47:12.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.913+0000 7fe11effd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:12.910 INFO:teuthology.orchestra.run.smithi007.stdout:47244640260 2024-05-08T15:47:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 >> v1:172.21.15.81:6812/847627118 conn(0x7fe1100399f0 legacy=0x7fe11003beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 10 monclient: shutdown 2024-05-08T15:47:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 >> v1:172.21.15.7:6793/0 conn(0x7fe1300a5780 legacy=0x7fe1301d1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 shutdown_connections 2024-05-08T15:47:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 >> 172.21.15.7:0/643826400 conn(0x7fe1300a9fa0 msgr2=0x7fe130140c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 shutdown_connections 2024-05-08T15:47:12.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.915+0000 7fe1349b2640 1 -- 172.21.15.7:0/643826400 wait complete. 2024-05-08T15:47:12.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.3 2024-05-08T15:47:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.972+0000 7f3ea6cb5640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-05-08T15:47:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.972+0000 7f3ea6cb5640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-05-08T15:47:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.972+0000 7f3ea6cb5640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.972+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3e80005370 con 0x7f3ea013e310 2024-05-08T15:47:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.973+0000 7f3e9d7fa640 1 -- 172.21.15.7:0/2506746124 <== mon.7 v1:172.21.15.81:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (unknown 1823589948 0 847495621) 0x7f3e980146d0 con 0x7f3ea013e310 2024-05-08T15:47:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.973+0000 7f3e9d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-05-08T15:47:12.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.973+0000 7f3e9d7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:12.970 INFO:teuthology.orchestra.run.smithi007.stdout:47244640259 2024-05-08T15:47:12.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 >> v1:172.21.15.81:6812/847627118 conn(0x7f3e8403a390 legacy=0x7f3e8403c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 10 monclient: shutdown 2024-05-08T15:47:12.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:12.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 >> v1:172.21.15.81:6792/0 conn(0x7f3ea013e310 legacy=0x7f3ea01d5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:12.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 shutdown_connections 2024-05-08T15:47:12.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 >> 172.21.15.7:0/2506746124 conn(0x7f3ea00a9fa0 msgr2=0x7f3ea00a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:12.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 shutdown_connections 2024-05-08T15:47:12.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:12.975+0000 7f3ea6cb5640 1 -- 172.21.15.7:0/2506746124 wait complete. 2024-05-08T15:47:12.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.4 2024-05-08T15:47:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.008+0000 7effaa8df640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-05-08T15:47:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.008+0000 7effaa8df640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-05-08T15:47:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.008+0000 7effaa8df640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.008+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7eff88005370 con 0x7effa4072710 2024-05-08T15:47:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.009+0000 7effa17fa640 1 -- 172.21.15.7:0/3301173576 <== mon.3 v1:172.21.15.81:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 847495621) 0x7eff940158d0 con 0x7effa4072710 2024-05-08T15:47:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.009+0000 7effa17fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-05-08T15:47:13.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.009+0000 7effa17fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:13.006 INFO:teuthology.orchestra.run.smithi007.stdout:47244640259 2024-05-08T15:47:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.011+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 >> v1:172.21.15.81:6812/847627118 conn(0x7eff8403a580 legacy=0x7eff8403ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.011+0000 7effaa8df640 10 monclient: shutdown 2024-05-08T15:47:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.011+0000 7effaa8df640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.011+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 >> v1:172.21.15.81:6790/0 conn(0x7effa4072710 legacy=0x7effa413dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:13.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.012+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 shutdown_connections 2024-05-08T15:47:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.012+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 >> 172.21.15.7:0/3301173576 conn(0x7effa40a7320 msgr2=0x7effa409b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.012+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 shutdown_connections 2024-05-08T15:47:13.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.012+0000 7effaa8df640 1 -- 172.21.15.7:0/3301173576 wait complete. 2024-05-08T15:47:13.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640259 for osd.2 2024-05-08T15:47:13.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.047+0000 7f3de8ff9640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-05-08T15:47:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.047+0000 7f3de8ff9640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-05-08T15:47:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.047+0000 7f3de8ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:47:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.047+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3df40008d0 con 0x7f3df40728b0 2024-05-08T15:47:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.047+0000 7f3deaffd640 1 -- 172.21.15.7:0/3392155336 <== mon.7 v1:172.21.15.81:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (unknown 2131975755 0 555833778) 0x7f3df0014600 con 0x7f3df40728b0 2024-05-08T15:47:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.047+0000 7f3deaffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-05-08T15:47:13.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.047+0000 7f3deaffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:13.044 INFO:teuthology.orchestra.run.smithi007.stdout:47244640258 2024-05-08T15:47:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.049+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 >> v1:172.21.15.81:6812/847627118 conn(0x7f3dd8039f50 legacy=0x7f3dd803c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.049+0000 7f3de8ff9640 10 monclient: shutdown 2024-05-08T15:47:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.049+0000 7f3de8ff9640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.049+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 >> v1:172.21.15.81:6792/0 conn(0x7f3df40728b0 legacy=0x7f3df41d51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.049+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 shutdown_connections 2024-05-08T15:47:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.049+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 >> 172.21.15.7:0/3392155336 conn(0x7f3df40a6aa0 msgr2=0x7f3df4070a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:13.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.050+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 shutdown_connections 2024-05-08T15:47:13.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.050+0000 7f3de8ff9640 1 -- 172.21.15.7:0/3392155336 wait complete. 2024-05-08T15:47:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.053+0000 7fde72c0f640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-05-08T15:47:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.053+0000 7fde72c0f640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-05-08T15:47:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.053+0000 7fde72c0f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.053+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fde4c005370 con 0x7fde6c06f010 2024-05-08T15:47:13.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.054+0000 7fde697fa640 1 -- 172.21.15.7:0/1044555450 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 555833778) 0x7fde58014520 con 0x7fde6c06f010 2024-05-08T15:47:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.054+0000 7fde697fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-05-08T15:47:13.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.054+0000 7fde697fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:13.051 INFO:teuthology.orchestra.run.smithi007.stdout:47244640258 2024-05-08T15:47:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 >> v1:172.21.15.81:6812/847627118 conn(0x7fde5003a410 legacy=0x7fde5003c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 10 monclient: shutdown 2024-05-08T15:47:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 >> v1:172.21.15.7:6793/0 conn(0x7fde6c06f010 legacy=0x7fde6c1d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 shutdown_connections 2024-05-08T15:47:13.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 >> 172.21.15.7:0/1044555450 conn(0x7fde6c0a6280 msgr2=0x7fde6c0755e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 shutdown_connections 2024-05-08T15:47:13.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:13.057+0000 7fde72c0f640 1 -- 172.21.15.7:0/1044555450 wait complete. 2024-05-08T15:47:13.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.5 2024-05-08T15:47:13.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640258 for osd.1 2024-05-08T15:47:13.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T15:47:13.924 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T15:47:13.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T15:47:14.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.012+0000 7f7d54e82640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.012+0000 7f7d54e82640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.012+0000 7f7d54e82640 10 monclient: monmap: 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.013930+0000 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.013930+0000 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.010 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.013+0000 7f7d54e82640 1 Processor -- start 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.013+0000 7f7d54e82640 1 -- start start 2024-05-08T15:47:14.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.013+0000 7f7d54e82640 10 monclient: init 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 10 monclient: _add_conns ranks=[1,3,7,5,0,4,6,2,8] 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 10 monclient(hunting): picked mon.noname-f con 0x7f7d500a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 10 monclient(hunting): picked mon.noname-g con 0x7f7d500a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 10 monclient(hunting): picked mon.noname-i con 0x7f7d5013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d500a1660 con 0x7f7d500a53a0 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d500a28f0 con 0x7f7d500a5780 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d500a2bd0 con 0x7f7d5013e310 2024-05-08T15:47:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d54e82640 10 monclient(hunting): authenticate will time out at 842.129151s 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d4ed9d640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7f7d5013e310 0x7f7d50140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:33106/0 (socket says 172.21.15.7:33106) 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d4ed9d640 1 -- 172.21.15.7:0/4001267280 learned_addr learned my addr 172.21.15.7:0/4001267280 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.014+0000 7f7d4dd9b640 1 --1- 172.21.15.7:0/4001267280 >> v1:172.21.15.81:6790/0 conn(0x7f7d500a5780 0x7f7d5013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50662/0 (socket says 172.21.15.7:50662) 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 989696251 0 0) 0x7f7d500a2bd0 con 0x7f7d5013e310 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): my global_id is 4336 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d38003520 con 0x7f7d5013e310 2024-05-08T15:47:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294527501 0 0) 0x7f7d500a1660 con 0x7f7d500a53a0 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): my global_id is 4312 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d500a2bd0 con 0x7f7d500a53a0 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1690368447 0 0) 0x7f7d500a28f0 con 0x7f7d500a5780 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient(hunting): my global_id is 74307 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d500a1660 con 0x7f7d500a5780 2024-05-08T15:47:14.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3870918479 0 0) 0x7f7d38003520 con 0x7f7d5013e310 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 >> v1:172.21.15.81:6790/0 conn(0x7f7d500a5780 legacy=0x7f7d5013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 >> v1:172.21.15.81:6789/0 conn(0x7f7d500a53a0 legacy=0x7f7d500a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 monclient: found mon.noname-i 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.81:6792/0 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d500a6730 con 0x7f7d5013e310 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7d48001b30 con 0x7f7d5013e310 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d54e82640 5 monclient: authenticate success, global_id 4336 2024-05-08T15:47:14.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d54e82640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: dump: 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.019 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d54e82640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 <== mon.7 v1:172.21.15.81:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7d48001d90 con 0x7f7d5013e310 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 1 -- 172.21.15.7:0/4001267280 <== mon.7 v1:172.21.15.81:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7d48004f60 con 0x7f7d5013e310 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d4d59a640 10 monclient: dump: 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.020 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d54e82640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.015+0000 7f7d54e82640 4 set_mon_vals no callback set 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.016+0000 7f7d54e82640 10 monclient: shutdown 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.016+0000 7f7d54e82640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.016+0000 7f7d54e82640 1 -- 172.21.15.7:0/4001267280 >> v1:172.21.15.81:6792/0 conn(0x7f7d5013e310 legacy=0x7f7d50140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.016+0000 7f7d54e82640 1 -- 172.21.15.7:0/4001267280 shutdown_connections 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.017+0000 7f7d54e82640 1 -- 172.21.15.7:0/4001267280 >> 172.21.15.7:0/4001267280 conn(0x7f7d500a9fa0 msgr2=0x7f7d500aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.017+0000 7f7d54e82640 1 -- 172.21.15.7:0/4001267280 shutdown_connections 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.017+0000 7f7d54e82640 1 -- 172.21.15.7:0/4001267280 wait complete. 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.018+0000 7f7d54e82640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.018+0000 7f7d54e82640 10 monclient: monmap: 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:14.022 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.018+0000 7f7d54e82640 1 Processor -- start 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.018+0000 7f7d54e82640 1 -- start start 2024-05-08T15:47:14.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.018+0000 7f7d54e82640 10 monclient: init 2024-05-08T15:47:14.024 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T15:47:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 10 monclient: _add_conns ranks=[3,7,8,1,0,2,6,5,4] 2024-05-08T15:47:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 10 monclient(hunting): picked mon.noname-g con 0x7f7d500a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 10 monclient(hunting): picked mon.noname-i con 0x7f7d500a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 10 monclient(hunting): picked mon.noname-e con 0x7f7d5013e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d50135240 con 0x7f7d500a53a0 2024-05-08T15:47:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d50134e20 con 0x7f7d500a5780 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d500a1e00 con 0x7f7d5013e310 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d54e82640 10 monclient(hunting): authenticate will time out at 842.134276s 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d4ed9d640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f7d5013e310 0x7f7d501d4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54550/0 (socket says 172.21.15.7:54550) 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.019+0000 7f7d4ed9d640 1 -- 172.21.15.7:0/3264385022 learned_addr learned my addr 172.21.15.7:0/3264385022 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 173637955 0 0) 0x7f7d500a1e00 con 0x7f7d5013e310 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): my global_id is 4310 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d30003520 con 0x7f7d5013e310 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2055832855 0 0) 0x7f7d50134e20 con 0x7f7d500a5780 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): my global_id is 4345 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d500a1e00 con 0x7f7d500a5780 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3567817203 0 0) 0x7f7d40001940 con 0x7f7d500a53a0 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): my global_id is 74316 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d50134e20 con 0x7f7d500a53a0 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1151856353 0 0) 0x7f7d30003520 con 0x7f7d5013e310 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d40001940 con 0x7f7d5013e310 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7d48004ea0 con 0x7f7d5013e310 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): dump: 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 470732841 0 0) 0x7f7d500a1e00 con 0x7f7d500a5780 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7d30003520 con 0x7f7d500a5780 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7d44001a50 con 0x7f7d500a5780 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient(hunting): dump: 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2833847689 0 0) 0x7f7d40001940 con 0x7f7d5013e310 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 >> v1:172.21.15.81:6792/0 conn(0x7f7d500a5780 legacy=0x7f7d5013c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 >> v1:172.21.15.81:6790/0 conn(0x7f7d500a53a0 legacy=0x7f7d5013bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 monclient: found mon.e 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d500a43c0 con 0x7f7d5013e310 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d54e82640 5 monclient: authenticate success, global_id 4310 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d54e82640 10 monclient: _renew_subs 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d54e82640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5013ca60 con 0x7f7d5013e310 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d54e82640 10 monclient: _renew_subs 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d54e82640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f7d5013d7e0 con 0x7f7d5013e310 2024-05-08T15:47:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7d48001d90 con 0x7f7d5013e310 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7d48001b10 con 0x7f7d5013e310 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.020+0000 7f7d3effd640 10 monclient: dump: 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.021+0000 7f7d3ffff640 4 set_mon_vals no callback set 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.021+0000 7f7d54e82640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.022+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7d480106f0 con 0x7f7d5013e310 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.022+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f7d48046e30 con 0x7f7d5013e310 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.022+0000 7f7d54e82640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.022+0000 7f7d54e82640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.022+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d34005080 con 0x7f7d5013e310 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.022+0000 7f7d4e59c640 10 monclient: get_auth_request con 0x7f7d3003a010 auth_method 0 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.027+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f7d480160e0 con 0x7f7d5013e310 2024-05-08T15:47:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.027+0000 7f7d3effd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.027+0000 7f7d3effd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.052+0000 7f3319ff4640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.052+0000 7f3319ff4640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.052+0000 7f3319ff4640 10 monclient: monmap: 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.053398+0000 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.053398+0000 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.050 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 1 Processor -- start 2024-05-08T15:47:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 1 -- start start 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient: init 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient: _add_conns ranks=[3,4,2,1,7,8,5,6,0] 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient(hunting): picked mon.noname-g con 0x7f33140a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient(hunting): picked mon.noname-c con 0x7f33140a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient(hunting): picked mon.noname-b con 0x7f331413e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33140a1660 con 0x7f331413e310 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33140a28f0 con 0x7f33140a53a0 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33140a2bd0 con 0x7f33140a5780 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.053+0000 7f3319ff4640 10 monclient(hunting): authenticate will time out at 842.168604s 2024-05-08T15:47:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f3313fff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f331413e310 0x7f3314140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44220/0 (socket says 172.21.15.7:44220) 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f3313fff640 1 -- 172.21.15.7:0/3321075754 learned_addr learned my addr 172.21.15.7:0/3321075754 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1934556172 0 0) 0x7f33140a1660 con 0x7f331413e310 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): my global_id is 74306 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f32fc003520 con 0x7f331413e310 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2247177115 0 0) 0x7f33140a28f0 con 0x7f33140a53a0 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): my global_id is 74325 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33140a1660 con 0x7f33140a53a0 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389387751 0 0) 0x7f33140a2bd0 con 0x7f33140a5780 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient(hunting): my global_id is 4315 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33140a28f0 con 0x7f33140a5780 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2366478331 0 0) 0x7f32fc003520 con 0x7f331413e310 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 >> v1:172.21.15.7:6791/0 conn(0x7f33140a5780 legacy=0x7f331413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 >> v1:172.21.15.81:6790/0 conn(0x7f33140a53a0 legacy=0x7f33140a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 monclient: found mon.noname-b 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33140a6730 con 0x7f331413e310 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f330c001b30 con 0x7f331413e310 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f33127fc640 10 monclient: dump: 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f3319ff4640 5 monclient: authenticate success, global_id 74306 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.054+0000 7f3319ff4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f330c001e50 con 0x7f331413e310 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f33127fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f33127fc640 1 -- 172.21.15.7:0/3321075754 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f330c004f60 con 0x7f331413e310 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f3319ff4640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f3319ff4640 4 set_mon_vals no callback set 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f33127fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f33127fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.055+0000 7f33127fc640 10 monclient: dump: 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.056+0000 7f3319ff4640 10 monclient: shutdown 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.056+0000 7f3319ff4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.056+0000 7f3319ff4640 1 -- 172.21.15.7:0/3321075754 >> v1:172.21.15.7:6790/0 conn(0x7f331413e310 legacy=0x7f3314140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.056+0000 7f3319ff4640 1 -- 172.21.15.7:0/3321075754 shutdown_connections 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.056+0000 7f3319ff4640 1 -- 172.21.15.7:0/3321075754 >> 172.21.15.7:0/3321075754 conn(0x7f33140a9fa0 msgr2=0x7f33140aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.056+0000 7f3319ff4640 1 -- 172.21.15.7:0/3321075754 shutdown_connections 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.056+0000 7f3319ff4640 1 -- 172.21.15.7:0/3321075754 wait complete. 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.057+0000 7f3319ff4640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.057+0000 7f3319ff4640 10 monclient: monmap: 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.062 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.057+0000 7f3319ff4640 1 Processor -- start 2024-05-08T15:47:14.062 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.057+0000 7f3319ff4640 1 -- start start 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.057+0000 7f3319ff4640 10 monclient: init 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 10 monclient: _add_conns ranks=[1,0,3,8,5,4,7,6,2] 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 10 monclient(hunting): picked mon.noname-f con 0x7f33140a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 10 monclient(hunting): picked mon.noname-a con 0x7f33140a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 10 monclient(hunting): picked mon.noname-g con 0x7f331413e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f331413d570 con 0x7f33140a5780 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3314134160 con 0x7f33140a53a0 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3314134440 con 0x7f331413e310 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3319ff4640 10 monclient(hunting): authenticate will time out at 842.173475s 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3312ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f33140a5780 0x7f3314137f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57358/0 (socket says 172.21.15.7:57358) 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.058+0000 7f3312ffd640 1 -- 172.21.15.7:0/1352017975 learned_addr learned my addr 172.21.15.7:0/1352017975 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465143815 0 0) 0x7f3314134160 con 0x7f33140a53a0 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): my global_id is 4321 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3300003520 con 0x7f33140a53a0 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364662428 0 0) 0x7f3314134440 con 0x7f331413e310 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): my global_id is 74334 2024-05-08T15:47:14.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3314134160 con 0x7f331413e310 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2013805342 0 0) 0x7f331413d570 con 0x7f33140a5780 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): my global_id is 4374 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3314134440 con 0x7f33140a5780 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1075129607 0 0) 0x7f3300003520 con 0x7f33140a53a0 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f331413d570 con 0x7f33140a53a0 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3304001a50 con 0x7f33140a53a0 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.069 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 562974983 0 0) 0x7f3314134160 con 0x7f331413e310 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3300003520 con 0x7f331413e310 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f330c004ea0 con 0x7f331413e310 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.071 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3102216225 0 0) 0x7f3314134440 con 0x7f33140a5780 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3314134160 con 0x7f33140a5780 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3308001990 con 0x7f33140a5780 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.072 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.073 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2364978853 0 0) 0x7f331413d570 con 0x7f33140a53a0 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 >> v1:172.21.15.81:6790/0 conn(0x7f331413e310 legacy=0x7f33141d4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 >> v1:172.21.15.7:6789/0 conn(0x7f33140a5780 legacy=0x7f3314137f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 monclient: found mon.f 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3314138660 con 0x7f33140a53a0 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f32fbfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f3319ff4640 5 monclient: authenticate success, global_id 4321 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f3319ff4640 10 monclient: _renew_subs 2024-05-08T15:47:14.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f3319ff4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3314133270 con 0x7f33140a53a0 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f3319ff4640 10 monclient: _renew_subs 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f3319ff4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.059+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f33141d5e30 con 0x7f33140a53a0 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.060+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3304001bf0 con 0x7f33140a53a0 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.060+0000 7f32fbfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.060+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3304003c00 con 0x7f33140a53a0 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.060+0000 7f32fbfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.060+0000 7f32fbfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.060+0000 7f32fbfff640 10 monclient: dump: 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.075 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.076 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.060+0000 7f3311ffb640 4 set_mon_vals no callback set 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.061+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f330400ef80 con 0x7f33140a53a0 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.061+0000 7f3319ff4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.061+0000 7f3319ff4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.061+0000 7f3319ff4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.061+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f4005080 con 0x7f33140a53a0 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.061+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f3304045930 con 0x7f33140a53a0 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.061+0000 7f3312ffd640 10 monclient: get_auth_request con 0x7f330003a620 auth_method 0 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.066+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f3304014ac0 con 0x7f33140a53a0 2024-05-08T15:47:14.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.066+0000 7f32fbfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.066+0000 7f32fbfff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:14.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.104+0000 7fbc08559640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.105+0000 7fbc08559640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.105+0000 7fbc08559640 10 monclient: monmap: 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.106033+0000 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.106033+0000 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.102 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.105+0000 7fbc08559640 1 Processor -- start 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.105+0000 7fbc08559640 1 -- start start 2024-05-08T15:47:14.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.105+0000 7fbc08559640 10 monclient: init 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc08559640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc08559640 10 monclient: _add_conns ranks=[1,2,4,6,5,0,7,8,3] 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc08559640 10 monclient(hunting): picked mon.noname-f con 0x7fbc000a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc08559640 10 monclient(hunting): picked mon.noname-b con 0x7fbc000a5780 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc08559640 10 monclient(hunting): picked mon.noname-c con 0x7fbc0013e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc08559640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc000a1660 con 0x7fbc000a53a0 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc05af4640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fbc000a5780 0x7fbc0013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:44222/0 (socket says 172.21.15.7:44222) 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.106+0000 7fbc05af4640 1 -- 172.21.15.7:0/4148617908 learned_addr learned my addr 172.21.15.7:0/4148617908 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc08559640 1 -- 172.21.15.7:0/4148617908 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc000a28f0 con 0x7fbc000a5780 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc08559640 1 -- 172.21.15.7:0/4148617908 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc000a2bd0 con 0x7fbc0013e310 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc08559640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2552126096 0 0) 0x7fbc000a1660 con 0x7fbc000a53a0 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc08559640 10 monclient(hunting): authenticate will time out at 842.222102s 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): my global_id is 4330 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbec003520 con 0x7fbc000a53a0 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 689564899 0 0) 0x7fbc000a2bd0 con 0x7fbc0013e310 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): my global_id is 4324 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc000a1660 con 0x7fbc0013e310 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2604266700 0 0) 0x7fbc000a28f0 con 0x7fbc000a5780 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 10 monclient(hunting): my global_id is 74315 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.107+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc000a2bd0 con 0x7fbc000a5780 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2820456237 0 0) 0x7fbbec003520 con 0x7fbc000a53a0 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 >> v1:172.21.15.7:6791/0 conn(0x7fbc0013e310 legacy=0x7fbc00140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 >> v1:172.21.15.7:6790/0 conn(0x7fbc000a5780 legacy=0x7fbc0013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 monclient: found mon.noname-f 2024-05-08T15:47:14.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc000a6730 con 0x7fbc000a53a0 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbbfc0017b0 con 0x7fbc000a53a0 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc08559640 5 monclient: authenticate success, global_id 4330 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc08559640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: dump: 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.122 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc08559640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbbfc001a80 con 0x7fbc000a53a0 2024-05-08T15:47:14.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 1 -- 172.21.15.7:0/4148617908 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbbfc003950 con 0x7fbc000a53a0 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc08559640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc08559640 4 set_mon_vals no callback set 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.108+0000 7fbc052f3640 10 monclient: dump: 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.124 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.109+0000 7fbc08559640 10 monclient: shutdown 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.109+0000 7fbc08559640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.109+0000 7fbc08559640 1 -- 172.21.15.7:0/4148617908 >> v1:172.21.15.81:6789/0 conn(0x7fbc000a53a0 legacy=0x7fbc000a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.109+0000 7fbc08559640 1 -- 172.21.15.7:0/4148617908 shutdown_connections 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.109+0000 7fbc08559640 1 -- 172.21.15.7:0/4148617908 >> 172.21.15.7:0/4148617908 conn(0x7fbc000a9fa0 msgr2=0x7fbc000aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.109+0000 7fbc08559640 1 -- 172.21.15.7:0/4148617908 shutdown_connections 2024-05-08T15:47:14.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.109+0000 7fbc08559640 1 -- 172.21.15.7:0/4148617908 wait complete. 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.110+0000 7fbc08559640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.110+0000 7fbc08559640 10 monclient: monmap: 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.126 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.110+0000 7fbc08559640 1 Processor -- start 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.110+0000 7fbc08559640 1 -- start start 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.110+0000 7fbc08559640 10 monclient: init 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 10 monclient: _add_conns ranks=[3,1,7,5,8,2,6,0,4] 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 10 monclient(hunting): picked mon.noname-g con 0x7fbc000a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 10 monclient(hunting): picked mon.noname-f con 0x7fbc000a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 10 monclient(hunting): picked mon.noname-i con 0x7fbc0013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc000a1e00 con 0x7fbc000a5780 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc0013ca70 con 0x7fbc000a53a0 2024-05-08T15:47:14.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc001344e0 con 0x7fbc0013e310 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc08559640 10 monclient(hunting): authenticate will time out at 842.226099s 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc06af6640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7fbc0013e310 0x7fbc001d4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:33114/0 (socket says 172.21.15.7:33114) 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc06af6640 1 -- 172.21.15.7:0/3804500827 learned_addr learned my addr 172.21.15.7:0/3804500827 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.111+0000 7fbc062f5640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fbc000a53a0 0x7fbc0013bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50680/0 (socket says 172.21.15.7:50680) 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2635543469 0 0) 0x7fbc001344e0 con 0x7fbc0013e310 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): my global_id is 4354 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbe4003520 con 0x7fbc0013e310 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215047536 0 0) 0x7fbc000a1e00 con 0x7fbc000a5780 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): my global_id is 4339 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc001344e0 con 0x7fbc000a5780 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3654204037 0 0) 0x7fbc0013ca70 con 0x7fbc000a53a0 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): my global_id is 74343 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc000a1e00 con 0x7fbc000a53a0 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3340695876 0 0) 0x7fbc001344e0 con 0x7fbc000a5780 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbc0013ca70 con 0x7fbc000a5780 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbbf0001a50 con 0x7fbc000a5780 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:14.129 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.130 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1286802428 0 0) 0x7fbc000a1e00 con 0x7fbc000a53a0 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbc001344e0 con 0x7fbc000a53a0 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbbfc0016b0 con 0x7fbc000a53a0 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.131 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2523024688 0 0) 0x7fbbe4003520 con 0x7fbc0013e310 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbc000a1e00 con 0x7fbc0013e310 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbbf8001d30 con 0x7fbc0013e310 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.133 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1253425741 0 0) 0x7fbc0013ca70 con 0x7fbc000a5780 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 >> v1:172.21.15.81:6792/0 conn(0x7fbc0013e310 legacy=0x7fbc001d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 >> v1:172.21.15.81:6790/0 conn(0x7fbc000a53a0 legacy=0x7fbc0013bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 monclient: found mon.f 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc001347c0 con 0x7fbc000a5780 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbbf6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbc08559640 5 monclient: authenticate success, global_id 4339 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbc08559640 10 monclient: _renew_subs 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbc08559640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.112+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbc00138750 con 0x7fbc000a5780 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbc08559640 10 monclient: _renew_subs 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbc08559640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fbc001337f0 con 0x7fbc000a5780 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbbf0004a90 con 0x7fbc000a5780 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbbf6ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbbf0004f80 con 0x7fbc000a5780 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbbf6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbbf6ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbbf6ffd640 10 monclient: dump: 2024-05-08T15:47:14.135 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.113+0000 7fbc04af2640 4 set_mon_vals no callback set 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.114+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fbbf0010300 con 0x7fbc000a5780 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.115+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fbbf0046c80 con 0x7fbc000a5780 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.115+0000 7fbc08559640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.115+0000 7fbc08559640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.115+0000 7fbc08559640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.115+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe8005080 con 0x7fbc000a5780 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.115+0000 7fbc062f5640 10 monclient: get_auth_request con 0x7fbbe403a4f0 auth_method 0 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.121+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fbbf0015d50 con 0x7fbc000a5780 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.121+0000 7fbbf6ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.121+0000 7fbbf6ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.160+0000 7fdd6dce4640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.160+0000 7fdd6dce4640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.160+0000 7fdd6dce4640 10 monclient: monmap: 2024-05-08T15:47:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.157 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.161799+0000 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.161799+0000 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.158 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.159 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.159 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.161+0000 7fdd6dce4640 1 Processor -- start 2024-05-08T15:47:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.161+0000 7fdd6dce4640 1 -- start start 2024-05-08T15:47:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.161+0000 7fdd6dce4640 10 monclient: init 2024-05-08T15:47:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 10 monclient: _add_conns ranks=[3,5,7,1,2,0,8,4,6] 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 10 monclient(hunting): picked mon.noname-g con 0x7fdd680a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 10 monclient(hunting): picked mon.noname-h con 0x7fdd680a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 10 monclient(hunting): picked mon.noname-i con 0x7fdd6813e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd680a1660 con 0x7fdd680a53a0 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd680a28f0 con 0x7fdd680a5780 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd680a2bd0 con 0x7fdd6813e310 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd6dce4640 10 monclient(hunting): authenticate will time out at 842.277285s 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd677fe640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fdd680a53a0 0x7fdd680a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50684/0 (socket says 172.21.15.7:50684) 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.162+0000 7fdd677fe640 1 -- 172.21.15.7:0/2950079016 learned_addr learned my addr 172.21.15.7:0/2950079016 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1337383391 0 0) 0x7fdd680a1660 con 0x7fdd680a53a0 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): my global_id is 74352 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd50003520 con 0x7fdd680a53a0 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2309115823 0 0) 0x7fdd680a28f0 con 0x7fdd680a5780 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): my global_id is 4361 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd680a1660 con 0x7fdd680a5780 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734599385 0 0) 0x7fdd680a2bd0 con 0x7fdd6813e310 2024-05-08T15:47:14.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient(hunting): my global_id is 4363 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd680a28f0 con 0x7fdd6813e310 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2302815348 0 0) 0x7fdd50003520 con 0x7fdd680a53a0 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 >> v1:172.21.15.81:6792/0 conn(0x7fdd6813e310 legacy=0x7fdd68140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 >> v1:172.21.15.81:6791/0 conn(0x7fdd680a5780 legacy=0x7fdd6813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 monclient: found mon.noname-g 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:14.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd680a6730 con 0x7fdd680a53a0 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdd580017b0 con 0x7fdd680a53a0 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: dump: 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.175 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.176 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd6dce4640 5 monclient: authenticate success, global_id 74352 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd6dce4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd58001a80 con 0x7fdd680a53a0 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 1 -- 172.21.15.7:0/2950079016 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdd58004be0 con 0x7fdd680a53a0 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd667fc640 10 monclient: dump: 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.177 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.178 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd6dce4640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.163+0000 7fdd6dce4640 4 set_mon_vals no callback set 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.164+0000 7fdd6dce4640 10 monclient: shutdown 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.164+0000 7fdd6dce4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.164+0000 7fdd6dce4640 1 -- 172.21.15.7:0/2950079016 >> v1:172.21.15.81:6790/0 conn(0x7fdd680a53a0 legacy=0x7fdd680a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.164+0000 7fdd6dce4640 1 -- 172.21.15.7:0/2950079016 shutdown_connections 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.164+0000 7fdd6dce4640 1 -- 172.21.15.7:0/2950079016 >> 172.21.15.7:0/2950079016 conn(0x7fdd680a9fa0 msgr2=0x7fdd680aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.164+0000 7fdd6dce4640 1 -- 172.21.15.7:0/2950079016 shutdown_connections 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.164+0000 7fdd6dce4640 1 -- 172.21.15.7:0/2950079016 wait complete. 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.165+0000 7fdd6dce4640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.165+0000 7fdd6dce4640 10 monclient: monmap: 2024-05-08T15:47:14.179 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.180 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.166+0000 7fdd6dce4640 1 Processor -- start 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.166+0000 7fdd6dce4640 1 -- start start 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.166+0000 7fdd6dce4640 10 monclient: init 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.166+0000 7fdd6dce4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.166+0000 7fdd6dce4640 10 monclient: _add_conns ranks=[8,6,1,5,7,2,0,4,3] 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.166+0000 7fdd6dce4640 10 monclient(hunting): picked mon.noname-e con 0x7fdd680a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd677fe640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fdd680a53a0 0x7fdd681ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54566/0 (socket says 172.21.15.7:54566) 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd66ffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fdd680a5780 0x7fdd681cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:60380/0 (socket says 172.21.15.7:60380) 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd677fe640 1 -- 172.21.15.7:0/3525074977 learned_addr learned my addr 172.21.15.7:0/3525074977 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd6dce4640 10 monclient(hunting): picked mon.noname-d con 0x7fdd680a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd6dce4640 10 monclient(hunting): picked mon.noname-f con 0x7fdd6813e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd68137af0 con 0x7fdd6813e310 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd681cdb30 con 0x7fdd680a5780 2024-05-08T15:47:14.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd680a43c0 con 0x7fdd680a53a0 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd6dce4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.167+0000 7fdd6dce4640 10 monclient(hunting): authenticate will time out at 842.282506s 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3850172198 0 0) 0x7fdd681cdb30 con 0x7fdd680a5780 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): my global_id is 74337 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd54003520 con 0x7fdd680a5780 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2975340348 0 0) 0x7fdd680a43c0 con 0x7fdd680a53a0 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): my global_id is 4319 2024-05-08T15:47:14.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd681cdb30 con 0x7fdd680a53a0 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705351807 0 0) 0x7fdd68137af0 con 0x7fdd6813e310 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): my global_id is 4348 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd680a43c0 con 0x7fdd6813e310 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 893790625 0 0) 0x7fdd54003520 con 0x7fdd680a5780 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd68137af0 con 0x7fdd680a5780 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdd5c001a40 con 0x7fdd680a5780 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.183 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.184 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2303186827 0 0) 0x7fdd680a43c0 con 0x7fdd6813e310 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd54003520 con 0x7fdd6813e310 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdd60001d30 con 0x7fdd6813e310 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.168+0000 7fdd4ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.185 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1043924003 0 0) 0x7fdd681cdb30 con 0x7fdd680a53a0 2024-05-08T15:47:14.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdd680a43c0 con 0x7fdd680a53a0 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdd580016b0 con 0x7fdd680a53a0 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.187 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 463674660 0 0) 0x7fdd68137af0 con 0x7fdd680a5780 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 >> v1:172.21.15.7:6793/0 conn(0x7fdd680a53a0 legacy=0x7fdd681ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 >> v1:172.21.15.81:6789/0 conn(0x7fdd6813e310 legacy=0x7fdd681d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 monclient: found mon.d 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd68133220 con 0x7fdd680a5780 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd4ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd6dce4640 5 monclient: authenticate success, global_id 74337 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd6dce4640 10 monclient: _renew_subs 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd6dce4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdd68134600 con 0x7fdd680a5780 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd6dce4640 10 monclient: _renew_subs 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd6dce4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.169+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fdd68133970 con 0x7fdd680a5780 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.170+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdd5c001ce0 con 0x7fdd680a5780 2024-05-08T15:47:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.170+0000 7fdd4ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.170+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdd5c003c00 con 0x7fdd680a5780 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.170+0000 7fdd4ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.170+0000 7fdd4ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.170+0000 7fdd4ffff640 10 monclient: dump: 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.190 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.170+0000 7fdd65ffb640 4 set_mon_vals no callback set 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.171+0000 7fdd6dce4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.171+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fdd5c003de0 con 0x7fdd680a5780 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.171+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fdd5c046c80 con 0x7fdd680a5780 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.171+0000 7fdd6dce4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.171+0000 7fdd6dce4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.171+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd48005080 con 0x7fdd680a5780 2024-05-08T15:47:14.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.171+0000 7fdd677fe640 10 monclient: get_auth_request con 0x7fdd54042da0 auth_method 0 2024-05-08T15:47:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.177+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fdd5c015e60 con 0x7fdd680a5780 2024-05-08T15:47:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.177+0000 7fdd4ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.177+0000 7fdd4ffff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:14.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.251+0000 7f77bd924640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.251+0000 7f77bd924640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.251+0000 7f77bd924640 10 monclient: monmap: 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.252223+0000 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.252223+0000 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.248 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.252+0000 7f7f0dec1640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.252+0000 7f7f0dec1640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.252+0000 7f7f0dec1640 10 monclient: monmap: 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.253115+0000 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.253115+0000 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.250 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.251 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.252+0000 7f77bd924640 1 Processor -- start 2024-05-08T15:47:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.252+0000 7f77bd924640 1 -- start start 2024-05-08T15:47:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.252+0000 7f77bd924640 10 monclient: init 2024-05-08T15:47:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 10 monclient: _add_conns ranks=[1,8,2,4,7,3,5,0,6] 2024-05-08T15:47:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 10 monclient(hunting): picked mon.noname-f con 0x7f77b80a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f7f0dec1640 1 Processor -- start 2024-05-08T15:47:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 10 monclient(hunting): picked mon.noname-e con 0x7f77b80a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 10 monclient(hunting): picked mon.noname-b con 0x7f77b813e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80a1660 con 0x7f77b80a53a0 2024-05-08T15:47:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f7f0dec1640 1 -- start start 2024-05-08T15:47:14.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f7f0dec1640 10 monclient: init 2024-05-08T15:47:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80a28f0 con 0x7f77b813e310 2024-05-08T15:47:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80a2bd0 con 0x7f77b80a5780 2024-05-08T15:47:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77bd924640 10 monclient(hunting): authenticate will time out at 842.367843s 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77b67fc640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f77b80a5780 0x7f77b813cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54572/0 (socket says 172.21.15.7:54572) 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77b67fc640 1 -- 172.21.15.7:0/2883617136 learned_addr learned my addr 172.21.15.7:0/2883617136 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3922531892 0 0) 0x7f77b80a1660 con 0x7f77b80a53a0 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77b5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77b5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77b5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.253+0000 7f77b5ffb640 10 monclient(hunting): my global_id is 4357 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a0003520 con 0x7f77b80a53a0 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405848734 0 0) 0x7f77b80a2bd0 con 0x7f77b80a5780 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient(hunting): my global_id is 4328 2024-05-08T15:47:14.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77b80a1660 con 0x7f77b80a5780 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 10 monclient: _add_conns ranks=[3,2,0,7,6,5,8,1,4] 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 10 monclient(hunting): picked mon.noname-g con 0x7f7f080671f0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 10 monclient(hunting): picked mon.noname-b con 0x7f7f0809cb30 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 10 monclient(hunting): picked mon.noname-a con 0x7f7f0809cf80 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0813c550 con 0x7f7f0809cf80 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0809a5f0 con 0x7f7f0809cb30 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0813cc50 con 0x7f7f080671f0 2024-05-08T15:47:14.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f0dec1640 10 monclient(hunting): authenticate will time out at 842.368914s 2024-05-08T15:47:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3665602255 0 0) 0x7f77a0003520 con 0x7f77b80a53a0 2024-05-08T15:47:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 >> v1:172.21.15.7:6793/0 conn(0x7f77b80a5780 legacy=0x7f77b813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 >> v1:172.21.15.7:6790/0 conn(0x7f77b813e310 legacy=0x7f77b8140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 monclient: found mon.noname-f 2024-05-08T15:47:14.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b80a6730 con 0x7f77b80a53a0 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77a80017b0 con 0x7f77b80a53a0 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: dump: 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77bd924640 5 monclient: authenticate success, global_id 4357 2024-05-08T15:47:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77bd924640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f07fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7f0809cf80 0x7f7f081447b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57372/0 (socket says 172.21.15.7:57372) 2024-05-08T15:47:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f7f07fff640 1 -- 172.21.15.7:0/2578665171 learned_addr learned my addr 172.21.15.7:0/2578665171 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f77a8001a10 con 0x7f77b80a53a0 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 1 -- 172.21.15.7:0/2883617136 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77a8003930 con 0x7f77b80a53a0 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77bd924640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77bd924640 4 set_mon_vals no callback set 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4091491978 0 0) 0x7f7f0813c550 con 0x7f7f0809cf80 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): my global_id is 4383 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ef4003520 con 0x7f7f0809cf80 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963252283 0 0) 0x7f7f0813cc50 con 0x7f7f080671f0 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient(hunting): my global_id is 74361 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ef40048e0 con 0x7f7f080671f0 2024-05-08T15:47:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.254+0000 7f77b5ffb640 10 monclient: dump: 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.263 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2643445223 0 0) 0x7f7ef4003520 con 0x7f7f0809cf80 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 >> v1:172.21.15.81:6790/0 conn(0x7f7f080671f0 legacy=0x7f7f0813be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 >> v1:172.21.15.7:6790/0 conn(0x7f7f0809cb30 legacy=0x7f7f08141080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T15:47:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f77bd924640 10 monclient: shutdown 2024-05-08T15:47:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f77bd924640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f77bd924640 1 -- 172.21.15.7:0/2883617136 >> v1:172.21.15.81:6789/0 conn(0x7f77b80a53a0 legacy=0x7f77b80a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f77bd924640 1 -- 172.21.15.7:0/2883617136 shutdown_connections 2024-05-08T15:47:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f77bd924640 1 -- 172.21.15.7:0/2883617136 >> 172.21.15.7:0/2883617136 conn(0x7f77b80a9fa0 msgr2=0x7f77b80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f77bd924640 1 -- 172.21.15.7:0/2883617136 shutdown_connections 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 monclient: found mon.noname-a 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f08099f30 con 0x7f7f0809cf80 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7f00001a40 con 0x7f7f0809cf80 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f06ffd640 10 monclient: dump: 2024-05-08T15:47:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f0dec1640 5 monclient: authenticate success, global_id 4383 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f7f0dec1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7f00001cf0 con 0x7f7f0809cf80 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f06ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f06ffd640 1 -- 172.21.15.7:0/2578665171 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7f00004c20 con 0x7f7f0809cf80 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f06ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f06ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f06ffd640 10 monclient: dump: 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.271 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f0dec1640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:14.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f0dec1640 4 set_mon_vals no callback set 2024-05-08T15:47:14.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.255+0000 7f77bd924640 1 -- 172.21.15.7:0/2883617136 wait complete. 2024-05-08T15:47:14.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f0dec1640 10 monclient: shutdown 2024-05-08T15:47:14.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f77bd924640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f77bd924640 10 monclient: monmap: 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.273 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.274 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.274 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f0dec1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.256+0000 7f7f0dec1640 1 -- 172.21.15.7:0/2578665171 >> v1:172.21.15.7:6789/0 conn(0x7f7f0809cf80 legacy=0x7f7f081447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.257+0000 7f77bd924640 1 Processor -- start 2024-05-08T15:47:14.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.257+0000 7f7f0dec1640 1 -- 172.21.15.7:0/2578665171 shutdown_connections 2024-05-08T15:47:14.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.257+0000 7f7f0dec1640 1 -- 172.21.15.7:0/2578665171 >> 172.21.15.7:0/2578665171 conn(0x7f7f080a2de0 msgr2=0x7f7f080a31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.257+0000 7f7f0dec1640 1 -- 172.21.15.7:0/2578665171 shutdown_connections 2024-05-08T15:47:14.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.257+0000 7f7f0dec1640 1 -- 172.21.15.7:0/2578665171 wait complete. 2024-05-08T15:47:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.257+0000 7f77bd924640 1 -- start start 2024-05-08T15:47:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.257+0000 7f77bd924640 10 monclient: init 2024-05-08T15:47:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f7f0dec1640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f7f0dec1640 10 monclient: monmap: 2024-05-08T15:47:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:14.276 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.277 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77bd924640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77bd924640 10 monclient: _add_conns ranks=[1,0,5,4,3,6,7,2,8] 2024-05-08T15:47:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77bd924640 10 monclient(hunting): picked mon.noname-f con 0x7f77b80a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:14.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77bd924640 10 monclient(hunting): picked mon.noname-a con 0x7f77b80a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77b6ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f77b80a53a0 0x7f77b81d9650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:53456/0 (socket says 172.21.15.7:53456) 2024-05-08T15:47:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77b67fc640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f77b80a5780 0x7f77b81de620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:57382/0 (socket says 172.21.15.7:57382) 2024-05-08T15:47:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77bd924640 10 monclient(hunting): picked mon.noname-h con 0x7f77b813e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f7f0dec1640 1 Processor -- start 2024-05-08T15:47:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77b6ffd640 1 -- 172.21.15.7:0/3437620990 learned_addr learned my addr 172.21.15.7:0/3437620990 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80a1e00 con 0x7f77b80a5780 2024-05-08T15:47:14.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f7f0dec1640 1 -- start start 2024-05-08T15:47:14.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.258+0000 7f7f0dec1640 10 monclient: init 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 111573026 0 0) 0x7f77b80a1e00 con 0x7f77b80a5780 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80a43c0 con 0x7f77b80a53a0 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b8071cb0 con 0x7f77b813e310 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f77bd924640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f77bd924640 10 monclient(hunting): authenticate will time out at 842.374026s 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 10 monclient: _add_conns ranks=[5,7,3,0,4,2,6,8,1] 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 10 monclient(hunting): picked mon.noname-h con 0x7f7f080671f0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 10 monclient(hunting): picked mon.noname-i con 0x7f7f0809cb30 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 10 monclient(hunting): picked mon.noname-g con 0x7f7f0809cf80 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0812d9e0 con 0x7f7f0809cf80 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0812dc60 con 0x7f7f080671f0 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f08099430 con 0x7f7f0809cb30 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f0dec1640 10 monclient(hunting): authenticate will time out at 842.374358s 2024-05-08T15:47:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f07fff640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f7f0809cf80 0x7f7f081d9020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:50698/0 (socket says 172.21.15.7:50698) 2024-05-08T15:47:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f07fff640 1 -- 172.21.15.7:0/594595923 learned_addr learned my addr 172.21.15.7:0/594595923 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f7f077fe640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7f7f080671f0 0x7f7f08137c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:36400/0 (socket says 172.21.15.7:36400) 2024-05-08T15:47:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): my global_id is 4392 2024-05-08T15:47:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a4003520 con 0x7f77b80a5780 2024-05-08T15:47:14.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2159691302 0 0) 0x7f77b80a43c0 con 0x7f77b80a53a0 2024-05-08T15:47:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1114029600 0 0) 0x7f7f08099430 con 0x7f7f0809cb30 2024-05-08T15:47:14.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): my global_id is 4372 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ee8003520 con 0x7f7f0809cb30 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029034415 0 0) 0x7f7f0812dc60 con 0x7f7f080671f0 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): my global_id is 4379 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f08099430 con 0x7f7f080671f0 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001050894 0 0) 0x7f7f08099f30 con 0x7f7f0809cf80 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 10 monclient(hunting): my global_id is 74370 2024-05-08T15:47:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f0812dc60 con 0x7f7f0809cf80 2024-05-08T15:47:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1635688449 0 0) 0x7f7f0812dc60 con 0x7f7f0809cf80 2024-05-08T15:47:14.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7ee8004d30 con 0x7f7f0809cf80 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): my global_id is 4366 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77b80a1e00 con 0x7f77b80a53a0 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2089865936 0 0) 0x7f77b8071cb0 con 0x7f77b813e310 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): my global_id is 4370 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77b80a43c0 con 0x7f77b813e310 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2772310928 0 0) 0x7f77b80a1e00 con 0x7f77b80a53a0 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77b8071cb0 con 0x7f77b80a53a0 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 797627662 0 0) 0x7f77a4003520 con 0x7f77b80a5780 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77b80a1e00 con 0x7f77b80a5780 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77a80016b0 con 0x7f77b80a53a0 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.259+0000 7f779ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.289 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:14.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77ac0019d0 con 0x7f77b80a5780 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.291 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2998993569 0 0) 0x7f77b80a43c0 con 0x7f77b813e310 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77a4003520 con 0x7f77b813e310 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77b0001cb0 con 0x7f77b813e310 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.292 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3340360119 0 0) 0x7f77b8071cb0 con 0x7f77b80a53a0 2024-05-08T15:47:14.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 >> v1:172.21.15.81:6791/0 conn(0x7f77b813e310 legacy=0x7f77b81e1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 >> v1:172.21.15.7:6789/0 conn(0x7f77b80a5780 legacy=0x7f77b81de620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 monclient: found mon.f 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b809be70 con 0x7f77b80a53a0 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77bd924640 5 monclient: authenticate success, global_id 4366 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77bd924640 10 monclient: _renew_subs 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77bd924640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f77b809b510 con 0x7f77b80a53a0 2024-05-08T15:47:14.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f77a8003c30 con 0x7f77b80a53a0 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77bd924640 10 monclient: _renew_subs 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77a80041a0 con 0x7f77b80a53a0 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77bd924640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f77b8071070 con 0x7f77b80a53a0 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.260+0000 7f77b57fa640 4 set_mon_vals no callback set 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f779ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f779ffff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f779ffff640 10 monclient: dump: 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.295 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.296 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7f00001ac0 con 0x7f7f0809cf80 2024-05-08T15:47:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:14.297 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.298 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.299 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.299 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3641756588 0 0) 0x7f7ee8004d30 con 0x7f7f0809cf80 2024-05-08T15:47:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f77a8004340 con 0x7f77b80a53a0 2024-05-08T15:47:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f77bd924640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f77bd924640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f77bd924640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 >> v1:172.21.15.81:6792/0 conn(0x7f7f0809cb30 legacy=0x7f7f08138330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 >> v1:172.21.15.81:6791/0 conn(0x7f7f080671f0 legacy=0x7f7f08137c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 monclient: found mon.g 2024-05-08T15:47:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:14.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f08138a40 con 0x7f7f0809cf80 2024-05-08T15:47:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f77a8046c50 con 0x7f77b80a53a0 2024-05-08T15:47:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b8075820 con 0x7f77b80a53a0 2024-05-08T15:47:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f04ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f0dec1640 5 monclient: authenticate success, global_id 74370 2024-05-08T15:47:14.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f77b67fc640 10 monclient: get_auth_request con 0x7f77a403a390 auth_method 0 2024-05-08T15:47:14.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f0dec1640 10 monclient: _renew_subs 2024-05-08T15:47:14.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f0dec1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:14.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0812bbb0 con 0x7f7f0809cf80 2024-05-08T15:47:14.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f0dec1640 10 monclient: _renew_subs 2024-05-08T15:47:14.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f0dec1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.261+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7f0812be30 con 0x7f7f0809cf80 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7f00001cf0 con 0x7f7f0809cf80 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f7f04ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7f00001ac0 con 0x7f7f0809cf80 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f7f04ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f7f04ff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f7f04ff9640 10 monclient: dump: 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.308 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.262+0000 7f7f067fc640 4 set_mon_vals no callback set 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.263+0000 7f7f0dec1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.263+0000 7f7f0dec1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.263+0000 7f7f0dec1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.263+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7f00004ee0 con 0x7f7f0809cf80 2024-05-08T15:47:14.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.263+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec005080 con 0x7f7f0809cf80 2024-05-08T15:47:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.264+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f7f00046a60 con 0x7f7f0809cf80 2024-05-08T15:47:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.264+0000 7f7f077fe640 10 monclient: get_auth_request con 0x7f7ee803a120 auth_method 0 2024-05-08T15:47:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.267+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f77a8015e60 con 0x7f77b80a53a0 2024-05-08T15:47:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.267+0000 7f779ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.267+0000 7f779ffff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.269+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f7f0812d9e0 con 0x7f7f0809cf80 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.269+0000 7f7f04ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.269+0000 7f7f04ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.297+0000 7f3319ff4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 3}] 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.297+0000 7f3319ff4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.297+0000 7f3319ff4640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.297+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f32f4005370 con 0x7f33140a53a0 2024-05-08T15:47:14.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.297+0000 7f32fbfff640 1 -- 172.21.15.7:0/1352017975 <== mon.1 v1:172.21.15.81:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (unknown 383520633 0 1902013079) 0x7f33040143c0 con 0x7f33140a53a0 2024-05-08T15:47:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.297+0000 7f32fbfff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 3}] 2024-05-08T15:47:14.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.297+0000 7f32fbfff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:14.312 INFO:teuthology.orchestra.run.smithi007.stdout:47244640262 2024-05-08T15:47:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.300+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 >> v1:172.21.15.81:6812/847627118 conn(0x7f330003a620 legacy=0x7f330003cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.300+0000 7f3319ff4640 10 monclient: shutdown 2024-05-08T15:47:14.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.300+0000 7f3319ff4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.300+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 >> v1:172.21.15.81:6789/0 conn(0x7f33140a53a0 legacy=0x7f33141c9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.300+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 shutdown_connections 2024-05-08T15:47:14.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.300+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 >> 172.21.15.7:0/1352017975 conn(0x7f33140a9fa0 msgr2=0x7f33140a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.301+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 shutdown_connections 2024-05-08T15:47:14.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.301+0000 7f3319ff4640 1 -- 172.21.15.7:0/1352017975 wait complete. 2024-05-08T15:47:14.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.3 2024-05-08T15:47:14.314 DEBUG:teuthology.parallel:result is None 2024-05-08T15:47:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.337+0000 7f7d54e82640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 0}] 2024-05-08T15:47:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.337+0000 7f7d54e82640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-05-08T15:47:14.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.337+0000 7f7d54e82640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.337+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7d34005370 con 0x7f7d5013e310 2024-05-08T15:47:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.338+0000 7f7d3effd640 1 -- 172.21.15.7:0/3264385022 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (unknown 574334944 0 2777258794) 0x7f7d480159e0 con 0x7f7d5013e310 2024-05-08T15:47:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.338+0000 7f7d3effd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 0}] 2024-05-08T15:47:14.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.338+0000 7f7d3effd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:14.335 INFO:teuthology.orchestra.run.smithi007.stdout:55834574853 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 >> v1:172.21.15.81:6812/847627118 conn(0x7f7d3003a010 legacy=0x7f7d3003c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 10 monclient: shutdown 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 >> v1:172.21.15.7:6793/0 conn(0x7f7d5013e310 legacy=0x7f7d501d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 shutdown_connections 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 >> 172.21.15.7:0/3264385022 conn(0x7f7d500a9fa0 msgr2=0x7f7d500a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 shutdown_connections 2024-05-08T15:47:14.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.341+0000 7f7d54e82640 1 -- 172.21.15.7:0/3264385022 wait complete. 2024-05-08T15:47:14.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-05-08T15:47:14.353 DEBUG:teuthology.parallel:result is None 2024-05-08T15:47:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.387+0000 7fdd6dce4640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 2}] 2024-05-08T15:47:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.387+0000 7fdd6dce4640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-05-08T15:47:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.387+0000 7fdd6dce4640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.387+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fdd48005370 con 0x7fdd680a5780 2024-05-08T15:47:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.387+0000 7fdd4ffff640 1 -- 172.21.15.7:0/3525074977 <== mon.6 v1:172.21.15.7:6792/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (unknown 92182286 0 1169801742) 0x7fdd5c015760 con 0x7fdd680a5780 2024-05-08T15:47:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.387+0000 7fdd4ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 2}] 2024-05-08T15:47:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.387+0000 7fdd4ffff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:14.384 INFO:teuthology.orchestra.run.smithi007.stdout:47244640261 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 >> v1:172.21.15.81:6812/847627118 conn(0x7fdd54042da0 legacy=0x7fdd54045260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 10 monclient: shutdown 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 >> v1:172.21.15.7:6792/0 conn(0x7fdd680a5780 legacy=0x7fdd681cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 shutdown_connections 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 >> 172.21.15.7:0/3525074977 conn(0x7fdd680a9fa0 msgr2=0x7fdd680a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 shutdown_connections 2024-05-08T15:47:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.389+0000 7fdd6dce4640 1 -- 172.21.15.7:0/3525074977 wait complete. 2024-05-08T15:47:14.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2024-05-08T15:47:14.397 DEBUG:teuthology.parallel:result is None 2024-05-08T15:47:14.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.453+0000 7fbc08559640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 4}] 2024-05-08T15:47:14.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.453+0000 7fbc08559640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-05-08T15:47:14.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.453+0000 7fbc08559640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.453+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fbbe8005370 con 0x7fbc000a5780 2024-05-08T15:47:14.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.454+0000 7fbbf6ffd640 1 -- 172.21.15.7:0/3804500827 <== mon.1 v1:172.21.15.81:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (unknown 1823589948 0 1169801742) 0x7fbbf0019a00 con 0x7fbc000a5780 2024-05-08T15:47:14.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.454+0000 7fbbf6ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 4}] 2024-05-08T15:47:14.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.454+0000 7fbbf6ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:14.451 INFO:teuthology.orchestra.run.smithi007.stdout:47244640261 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 >> v1:172.21.15.81:6812/847627118 conn(0x7fbbe403a4f0 legacy=0x7fbbe403c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 10 monclient: shutdown 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 >> v1:172.21.15.81:6789/0 conn(0x7fbc000a5780 legacy=0x7fbc0013c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 shutdown_connections 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 >> 172.21.15.7:0/3804500827 conn(0x7fbc000a9fa0 msgr2=0x7fbc000a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 shutdown_connections 2024-05-08T15:47:14.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.456+0000 7fbc08559640 1 -- 172.21.15.7:0/3804500827 wait complete. 2024-05-08T15:47:14.463 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.4 2024-05-08T15:47:14.463 DEBUG:teuthology.parallel:result is None 2024-05-08T15:47:14.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.500+0000 7f77bd924640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 5}] 2024-05-08T15:47:14.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.500+0000 7f77bd924640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-05-08T15:47:14.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.500+0000 7f77bd924640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:47:14.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.500+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f77b806b380 con 0x7f77b80a53a0 2024-05-08T15:47:14.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.500+0000 7f779ffff640 1 -- 172.21.15.7:0/3437620990 <== mon.1 v1:172.21.15.81:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (unknown 2131975755 0 1444633209) 0x7f77a8015760 con 0x7f77b80a53a0 2024-05-08T15:47:14.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.500+0000 7f779ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 5}] 2024-05-08T15:47:14.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.500+0000 7f779ffff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:14.498 INFO:teuthology.orchestra.run.smithi007.stdout:47244640260 2024-05-08T15:47:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 >> v1:172.21.15.81:6812/847627118 conn(0x7f77a403a390 legacy=0x7f77a403c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 10 monclient: shutdown 2024-05-08T15:47:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 >> v1:172.21.15.81:6789/0 conn(0x7f77b80a53a0 legacy=0x7f77b81d9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 shutdown_connections 2024-05-08T15:47:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 >> 172.21.15.7:0/3437620990 conn(0x7f77b80a9fa0 msgr2=0x7f77b813b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 shutdown_connections 2024-05-08T15:47:14.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.502+0000 7f77bd924640 1 -- 172.21.15.7:0/3437620990 wait complete. 2024-05-08T15:47:14.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.5 2024-05-08T15:47:14.510 DEBUG:teuthology.parallel:result is None 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.533+0000 7f7f0dec1640 10 start_mon_command cmd=[{"prefix": "osd last-stat-seq", "id": 1}] 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.533+0000 7f7f0dec1640 10 monclient: _send_command 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.533+0000 7f7f0dec1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.533+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7eec005370 con 0x7f7f0809cf80 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.534+0000 7f7f04ff9640 1 -- 172.21.15.7:0/594595923 <== mon.3 v1:172.21.15.81:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (unknown 832126871 0 1444633209) 0x7f7f00015c50 con 0x7f7f0809cf80 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.534+0000 7f7f04ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd last-stat-seq", "id": 1}] 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.534+0000 7f7f04ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:14.531 INFO:teuthology.orchestra.run.smithi007.stdout:47244640260 2024-05-08T15:47:14.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 >> v1:172.21.15.81:6812/847627118 conn(0x7f7ee803a120 legacy=0x7f7ee803c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 10 monclient: shutdown 2024-05-08T15:47:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 >> v1:172.21.15.81:6790/0 conn(0x7f7f0809cf80 legacy=0x7f7f081d9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 shutdown_connections 2024-05-08T15:47:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 >> 172.21.15.7:0/594595923 conn(0x7f7f080a2de0 msgr2=0x7f7f0809aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 shutdown_connections 2024-05-08T15:47:14.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.536+0000 7f7f0dec1640 1 -- 172.21.15.7:0/594595923 wait complete. 2024-05-08T15:47:14.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640260 got 47244640260 for osd.1 2024-05-08T15:47:14.543 DEBUG:teuthology.parallel:result is None 2024-05-08T15:47:14.543 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T15:47:14.543 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:47:14.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.689+0000 7f02ea64e640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.689+0000 7f02ea64e640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.689+0000 7f02ea64e640 10 monclient: monmap: 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.690948+0000 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.690948+0000 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.687 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.688 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.688 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.690+0000 7f02ea64e640 1 Processor -- start 2024-05-08T15:47:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.690+0000 7f02ea64e640 1 -- start start 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.690+0000 7f02ea64e640 10 monclient: init 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 10 monclient: _add_conns ranks=[6,2,0,5,3,8,4,7,1] 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 10 monclient(hunting): picked mon.noname-d con 0x7f02e4137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 10 monclient(hunting): picked mon.noname-b con 0x7f02e40a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 10 monclient(hunting): picked mon.noname-a con 0x7f02e40a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e40a2c70 con 0x7f02e40a41b0 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e40a2f50 con 0x7f02e40a49a0 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e40a1670 con 0x7f02e4137940 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02ea64e640 10 monclient(hunting): authenticate will time out at 842.806065s 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e3fff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f02e4137940 0x7f02e4137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56924/0 (socket says 172.21.15.7:56924) 2024-05-08T15:47:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e3fff640 1 -- 172.21.15.7:0/1501625719 learned_addr learned my addr 172.21.15.7:0/1501625719 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02dbfff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f02e40a49a0 0x7f02e413cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:32882/0 (socket says 172.21.15.7:32882) 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1406883597 0 0) 0x7f02e40a1670 con 0x7f02e4137940 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): my global_id is 74346 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02d0003520 con 0x7f02e4137940 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3298494557 0 0) 0x7f02e40a2f50 con 0x7f02e40a49a0 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): my global_id is 74333 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02e40a1670 con 0x7f02e40a49a0 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1770265422 0 0) 0x7f02e40a2c70 con 0x7f02e40a41b0 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 10 monclient(hunting): my global_id is 4401 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.691+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02e40a2f50 con 0x7f02e40a41b0 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2120714996 0 0) 0x7f02e40a1670 con 0x7f02e40a49a0 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 >> v1:172.21.15.7:6792/0 conn(0x7f02e4137940 legacy=0x7f02e4137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 >> v1:172.21.15.7:6789/0 conn(0x7f02e40a41b0 legacy=0x7f02e4140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 monclient: found mon.noname-b 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e40a39f0 con 0x7f02e40a49a0 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f02cc001710 con 0x7f02e40a49a0 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: dump: 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 5 monclient: authenticate success, global_id 74333 2024-05-08T15:47:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f02cc0019f0 con 0x7f02e40a49a0 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 1 -- 172.21.15.7:0/1501625719 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f02cc004b20 con 0x7f02e40a49a0 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 4 set_mon_vals no callback set 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02e37fe640 10 monclient: dump: 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 10 monclient: shutdown 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 1 -- 172.21.15.7:0/1501625719 >> v1:172.21.15.7:6790/0 conn(0x7f02e40a49a0 legacy=0x7f02e413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 1 -- 172.21.15.7:0/1501625719 shutdown_connections 2024-05-08T15:47:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.692+0000 7f02ea64e640 1 -- 172.21.15.7:0/1501625719 >> 172.21.15.7:0/1501625719 conn(0x7f02e40a9fa0 msgr2=0x7f02e40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.693+0000 7f02ea64e640 1 -- 172.21.15.7:0/1501625719 shutdown_connections 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.693+0000 7f02ea64e640 1 -- 172.21.15.7:0/1501625719 wait complete. 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.693+0000 7f02ea64e640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.693+0000 7f02ea64e640 10 monclient: monmap: 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.693+0000 7f02ea64e640 1 Processor -- start 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.693+0000 7f02ea64e640 1 -- start start 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.693+0000 7f02ea64e640 10 monclient: init 2024-05-08T15:47:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 10 monclient: _add_conns ranks=[6,3,8,1,7,5,2,4,0] 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 10 monclient(hunting): picked mon.noname-d con 0x7f02e40a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 10 monclient(hunting): picked mon.noname-g con 0x7f02e40a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 10 monclient(hunting): picked mon.noname-e con 0x7f02e4137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e40a0f60 con 0x7f02e40a49a0 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e41352a0 con 0x7f02e40a41b0 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e4135580 con 0x7f02e4137940 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02ea64e640 10 monclient(hunting): authenticate will time out at 842.809026s 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e3fff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f02e40a41b0 0x7f02e4135f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56926/0 (socket says 172.21.15.7:56926) 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e3fff640 1 -- 172.21.15.7:0/4101017063 learned_addr learned my addr 172.21.15.7:0/4101017063 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2107706568 0 0) 0x7f02e4135580 con 0x7f02e4137940 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): my global_id is 4337 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02c4003520 con 0x7f02e4137940 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1499414892 0 0) 0x7f02e41352a0 con 0x7f02e40a41b0 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 10 monclient(hunting): my global_id is 74355 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02e4135580 con 0x7f02e40a41b0 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.694+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2174620357 0 0) 0x7f02c4003520 con 0x7f02e4137940 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f02e41352a0 con 0x7f02e4137940 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3244399281 0 0) 0x7f02e40a0f60 con 0x7f02e40a49a0 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): my global_id is 74379 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02c4003520 con 0x7f02e40a49a0 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f02dc001690 con 0x7f02e4137940 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): dump: 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2596095896 0 0) 0x7f02e4135580 con 0x7f02e40a41b0 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f02e40a0f60 con 0x7f02e40a41b0 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f02d4001a10 con 0x7f02e40a41b0 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient(hunting): dump: 2024-05-08T15:47:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4188589656 0 0) 0x7f02e41352a0 con 0x7f02e4137940 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 >> v1:172.21.15.7:6792/0 conn(0x7f02e40a41b0 legacy=0x7f02e4135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 >> v1:172.21.15.81:6790/0 conn(0x7f02e40a49a0 legacy=0x7f02e41d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: _finish_hunting 0 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 monclient: found mon.e 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e4135860 con 0x7f02e4137940 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: _finish_auth 0 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: _check_auth_tickets 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02ea64e640 5 monclient: authenticate success, global_id 4337 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02ea64e640 10 monclient: _renew_subs 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02ea64e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f02e4133860 con 0x7f02e4137940 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02ea64e640 10 monclient: _renew_subs 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02ea64e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f02e41d5a90 con 0x7f02e4137940 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f02dc0019d0 con 0x7f02e4137940 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f02dc003f60 con 0x7f02e4137940 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e2ffd640 4 set_mon_vals no callback set 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.695+0000 7f02e17fa640 10 monclient: dump: 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.696+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f02dc00f2e0 con 0x7f02e4137940 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.696+0000 7f02ea64e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.696+0000 7f02ea64e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.696+0000 7f02ea64e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.696+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c8005080 con 0x7f02e4137940 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.696+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f02dc045cf0 con 0x7f02e4137940 2024-05-08T15:47:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.696+0000 7f02e3fff640 10 monclient: get_auth_request con 0x7f02c403a0c0 auth_method 0 2024-05-08T15:47:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.700+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f02dc014e20 con 0x7f02e4137940 2024-05-08T15:47:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.700+0000 7f02e17fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.700+0000 7f02e17fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.879+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02c8000b90 con 0x7f02c403a0c0 2024-05-08T15:47:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.881+0000 7f02e17fa640 1 -- 172.21.15.7:0/4101017063 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33876 (unknown 2965378022 0 4059739380) 0x7f02c8000b90 con 0x7f02c403a0c0 2024-05-08T15:47:14.878 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:47:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 >> v1:172.21.15.81:6812/847627118 conn(0x7f02c403a0c0 legacy=0x7f02c403c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 10 monclient: shutdown 2024-05-08T15:47:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 >> v1:172.21.15.7:6793/0 conn(0x7f02e4137940 legacy=0x7f02e41d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 shutdown_connections 2024-05-08T15:47:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 >> 172.21.15.7:0/4101017063 conn(0x7f02e40a9fa0 msgr2=0x7f02e4130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 shutdown_connections 2024-05-08T15:47:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.883+0000 7f02ea64e640 1 -- 172.21.15.7:0/4101017063 wait complete. 2024-05-08T15:47:14.890 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-05-08T15:47:13.097938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167256,"kb_used_data":968,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566063784,"statfs":{"total":579820584960,"available":579649314816,"internally_reserved":0,"allocated":991232,"data_stored":1320416,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":170187426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000865"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172443+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:47:12.172443+0000","last_peered":"2024-05-08T15:47:12.172443+0000","last_clean":"2024-05-08T15:47:12.172443+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:47:12.172443+0000","last_undegraded":"2024-05-08T15:47:12.172443+0000","last_fullsized":"2024-05-08T15:47:12.172443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:14.941756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172367+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:47:12.172367+0000","last_peered":"2024-05-08T15:47:12.172367+0000","last_clean":"2024-05-08T15:47:12.172367+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:47:12.172367+0000","last_undegraded":"2024-05-08T15:47:12.172367+0000","last_fullsized":"2024-05-08T15:47:12.172367+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:28.945664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.173944+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:30.537193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171573+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:47:12.171573+0000","last_peered":"2024-05-08T15:47:12.171573+0000","last_clean":"2024-05-08T15:47:12.171573+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:47:12.171573+0000","last_undegraded":"2024-05-08T15:47:12.171573+0000","last_fullsized":"2024-05-08T15:47:12.171573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171867+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:47:12.171867+0000","last_peered":"2024-05-08T15:47:12.171867+0000","last_clean":"2024-05-08T15:47:12.171867+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:47:12.171867+0000","last_undegraded":"2024-05-08T15:47:12.171867+0000","last_fullsized":"2024-05-08T15:47:12.171867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171427+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:47:12.171427+0000","last_peered":"2024-05-08T15:47:12.171427+0000","last_clean":"2024-05-08T15:47:12.171427+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:47:12.171427+0000","last_undegraded":"2024-05-08T15:47:12.171427+0000","last_fullsized":"2024-05-08T15:47:12.171427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:49.677393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171045+0000","last_change":"2024-05-08T15:47:11.176829+0000","last_active":"2024-05-08T15:47:12.171045+0000","last_peered":"2024-05-08T15:47:12.171045+0000","last_clean":"2024-05-08T15:47:12.171045+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:47:12.171045+0000","last_undegraded":"2024-05-08T15:47:12.171045+0000","last_fullsized":"2024-05-08T15:47:12.171045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:32.476618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:30.243149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172345+0000","last_change":"2024-05-08T15:47:04.948326+0000","last_active":"2024-05-08T15:47:12.172345+0000","last_peered":"2024-05-08T15:47:12.172345+0000","last_clean":"2024-05-08T15:47:12.172345+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:47:12.172345+0000","last_undegraded":"2024-05-08T15:47:12.172345+0000","last_fullsized":"2024-05-08T15:47:12.172345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_clean_scrub_stamp":"2024-05-08T15:47:03.939015+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:25.024655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31324,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340516,"statfs":{"total":96636764160,"available":96604688384,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:47:14.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:47:14.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.998+0000 7f1b221bd640 10 monclient: get_monmap_and_config 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.998+0000 7f1b221bd640 10 monclient: build_initial_monmap 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.998+0000 7f1b221bd640 10 monclient: monmap: 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:14.999540+0000 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:14.999540+0000 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:14.995 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:14.996 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:14.996 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 1 Processor -- start 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 1 -- start start 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient: init 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient: _add_conns ranks=[4,0,7,8,3,6,5,2,1] 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient(hunting): picked mon.noname-c con 0x7f1b1c137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient(hunting): picked mon.noname-a con 0x7f1b1c0a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient(hunting): picked mon.noname-i con 0x7f1b1c0a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:15.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c0a2c70 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c0a2f50 con 0x7f1b1c137940 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c0a1670 con 0x7f1b1c0a41b0 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b221bd640 10 monclient(hunting): authenticate will time out at 843.114479s 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b1b7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f1b1c137940 0x7f1b1c137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:41062/0 (socket says 172.21.15.7:41062) 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b1b7fe640 1 -- 172.21.15.7:0/1234755941 learned_addr learned my addr 172.21.15.7:0/1234755941 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:14.999+0000 7f1b1affd640 1 --1- 172.21.15.7:0/1234755941 >> v1:172.21.15.7:6789/0 conn(0x7f1b1c0a49a0 0x7f1b1c13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58114/0 (socket says 172.21.15.7:58114) 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422429284 0 0) 0x7f1b1c0a2f50 con 0x7f1b1c137940 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): my global_id is 4333 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b04003520 con 0x7f1b1c137940 2024-05-08T15:47:15.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2436845901 0 0) 0x7f1b1c0a2c70 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): my global_id is 4410 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b1c0a2f50 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323596650 0 0) 0x7f1b1c0a1670 con 0x7f1b1c0a41b0 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient(hunting): my global_id is 4381 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b1c0a2c70 con 0x7f1b1c0a41b0 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3733688479 0 0) 0x7f1b04003520 con 0x7f1b1c137940 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 >> v1:172.21.15.81:6792/0 conn(0x7f1b1c0a41b0 legacy=0x7f1b1c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 >> v1:172.21.15.7:6789/0 conn(0x7f1b1c0a49a0 legacy=0x7f1b1c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 monclient: found mon.noname-c 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c0a39f0 con 0x7f1b1c137940 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b221bd640 5 monclient: authenticate success, global_id 4333 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1b0c0017b0 con 0x7f1b1c137940 2024-05-08T15:47:15.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: dump: 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.008 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b221bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1b0c001a10 con 0x7f1b1c137940 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 1 -- 172.21.15.7:0/1234755941 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1b0c004be0 con 0x7f1b1c137940 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b1a7fc640 10 monclient: dump: 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.009 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.010 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b221bd640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.000+0000 7f1b221bd640 4 set_mon_vals no callback set 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 10 monclient: shutdown 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 1 -- 172.21.15.7:0/1234755941 >> v1:172.21.15.7:6791/0 conn(0x7f1b1c137940 legacy=0x7f1b1c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 1 -- 172.21.15.7:0/1234755941 shutdown_connections 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 1 -- 172.21.15.7:0/1234755941 >> 172.21.15.7:0/1234755941 conn(0x7f1b1c0a9fa0 msgr2=0x7f1b1c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 1 -- 172.21.15.7:0/1234755941 shutdown_connections 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 1 -- 172.21.15.7:0/1234755941 wait complete. 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 10 monclient: build_initial_monmap 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.001+0000 7f1b221bd640 10 monclient: monmap: 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:15.011 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 1 Processor -- start 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 1 -- start start 2024-05-08T15:47:15.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient: init 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient: _add_conns ranks=[5,1,6,0,3,8,2,4,7] 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient(hunting): picked mon.noname-h con 0x7f1b1c0a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient(hunting): picked mon.noname-f con 0x7f1b1c0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient(hunting): picked mon.noname-d con 0x7f1b1c137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c135910 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c132b80 con 0x7f1b1c0a41b0 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b1c0a1f30 con 0x7f1b1c137940 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b221bd640 10 monclient(hunting): authenticate will time out at 843.117433s 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b1bfff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f1b1c137940 0x7f1b1c1d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56932/0 (socket says 172.21.15.7:56932) 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.002+0000 7f1b1bfff640 1 -- 172.21.15.7:0/2786266845 learned_addr learned my addr 172.21.15.7:0/2786266845 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2436996795 0 0) 0x7f1b1c0a1f30 con 0x7f1b1c137940 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): my global_id is 74364 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b08003520 con 0x7f1b1c137940 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2495652258 0 0) 0x7f1b1c135910 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): my global_id is 4375 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b1c0a1f30 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3604575819 0 0) 0x7f1b1c132b80 con 0x7f1b1c0a41b0 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): my global_id is 4388 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b1c135910 con 0x7f1b1c0a41b0 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2491403807 0 0) 0x7f1b08003520 con 0x7f1b1c137940 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1b1c132b80 con 0x7f1b1c137940 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1b14001df0 con 0x7f1b1c137940 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): dump: 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 175099428 0 0) 0x7f1b1c0a1f30 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1b08003520 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1b10001510 con 0x7f1b1c0a49a0 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient(hunting): dump: 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.017 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3788663357 0 0) 0x7f1b1c132b80 con 0x7f1b1c137940 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 >> v1:172.21.15.81:6791/0 conn(0x7f1b1c0a41b0 legacy=0x7f1b1c131d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 >> v1:172.21.15.81:6789/0 conn(0x7f1b1c0a49a0 legacy=0x7f1b1c132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 monclient: found mon.d 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c134e90 con 0x7f1b1c137940 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: _finish_auth 0 2024-05-08T15:47:15.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b221bd640 5 monclient: authenticate success, global_id 74364 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b221bd640 10 monclient: _renew_subs 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b221bd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c135630 con 0x7f1b1c137940 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b221bd640 10 monclient: _renew_subs 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b221bd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1375a0 con 0x7f1b1c137940 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1b14003c40 con 0x7f1b1c137940 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1b14004130 con 0x7f1b1c137940 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b03fff640 10 monclient: dump: 2024-05-08T15:47:15.019 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.020 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.003+0000 7f1b19ffb640 4 set_mon_vals no callback set 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.004+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f1b14004350 con 0x7f1b1c137940 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.004+0000 7f1b221bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.004+0000 7f1b221bd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.004+0000 7f1b221bd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.004+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc005080 con 0x7f1b1c137940 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.004+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f1b14047020 con 0x7f1b1c137940 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.005+0000 7f1b1b7fe640 10 monclient: get_auth_request con 0x7f1b0803a000 auth_method 0 2024-05-08T15:47:15.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.009+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f1b14016290 con 0x7f1b1c137940 2024-05-08T15:47:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.009+0000 7f1b03fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.009+0000 7f1b03fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.194+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1afc000b90 con 0x7f1b0803a000 2024-05-08T15:47:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.196+0000 7f1b03fff640 1 -- 172.21.15.7:0/2786266845 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33864 (unknown 2965378022 0 2814674891) 0x7f1afc000b90 con 0x7f1b0803a000 2024-05-08T15:47:15.193 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:47:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 >> v1:172.21.15.81:6812/847627118 conn(0x7f1b0803a000 legacy=0x7f1b0803c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 10 monclient: shutdown 2024-05-08T15:47:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 >> v1:172.21.15.7:6792/0 conn(0x7f1b1c137940 legacy=0x7f1b1c1d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 shutdown_connections 2024-05-08T15:47:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 >> 172.21.15.7:0/2786266845 conn(0x7f1b1c0a9fa0 msgr2=0x7f1b1c130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 shutdown_connections 2024-05-08T15:47:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.198+0000 7f1b221bd640 1 -- 172.21.15.7:0/2786266845 wait complete. 2024-05-08T15:47:15.206 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-08T15:47:15.098479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167268,"kb_used_data":976,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566063772,"statfs":{"total":579820584960,"available":579649302528,"internally_reserved":0,"allocated":999424,"data_stored":1324460,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":170187426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172443+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:47:12.172443+0000","last_peered":"2024-05-08T15:47:12.172443+0000","last_clean":"2024-05-08T15:47:12.172443+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:47:12.172443+0000","last_undegraded":"2024-05-08T15:47:12.172443+0000","last_fullsized":"2024-05-08T15:47:12.172443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:14.941756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172367+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:47:12.172367+0000","last_peered":"2024-05-08T15:47:12.172367+0000","last_clean":"2024-05-08T15:47:12.172367+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:47:12.172367+0000","last_undegraded":"2024-05-08T15:47:12.172367+0000","last_fullsized":"2024-05-08T15:47:12.172367+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:28.945664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.173944+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:30.537193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171573+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:47:12.171573+0000","last_peered":"2024-05-08T15:47:12.171573+0000","last_clean":"2024-05-08T15:47:12.171573+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:47:12.171573+0000","last_undegraded":"2024-05-08T15:47:12.171573+0000","last_fullsized":"2024-05-08T15:47:12.171573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171867+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:47:12.171867+0000","last_peered":"2024-05-08T15:47:12.171867+0000","last_clean":"2024-05-08T15:47:12.171867+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:47:12.171867+0000","last_undegraded":"2024-05-08T15:47:12.171867+0000","last_fullsized":"2024-05-08T15:47:12.171867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171427+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:47:12.171427+0000","last_peered":"2024-05-08T15:47:12.171427+0000","last_clean":"2024-05-08T15:47:12.171427+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:47:12.171427+0000","last_undegraded":"2024-05-08T15:47:12.171427+0000","last_fullsized":"2024-05-08T15:47:12.171427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:49.677393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171045+0000","last_change":"2024-05-08T15:47:11.176829+0000","last_active":"2024-05-08T15:47:12.171045+0000","last_peered":"2024-05-08T15:47:12.171045+0000","last_clean":"2024-05-08T15:47:12.171045+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:47:12.171045+0000","last_undegraded":"2024-05-08T15:47:12.171045+0000","last_fullsized":"2024-05-08T15:47:12.171045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:32.476618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:30.243149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172345+0000","last_change":"2024-05-08T15:47:04.948326+0000","last_active":"2024-05-08T15:47:12.172345+0000","last_peered":"2024-05-08T15:47:12.172345+0000","last_clean":"2024-05-08T15:47:12.172345+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:47:12.172345+0000","last_undegraded":"2024-05-08T15:47:12.172345+0000","last_fullsized":"2024-05-08T15:47:12.172345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_clean_scrub_stamp":"2024-05-08T15:47:03.939015+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:25.024655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:47:15.207 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T15:47:15.207 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T15:47:15.207 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T15:47:15.207 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T15:47:15.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.312+0000 7fb6679bd640 10 monclient: get_monmap_and_config 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.312+0000 7fb6679bd640 10 monclient: build_initial_monmap 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.312+0000 7fb6679bd640 10 monclient: monmap: 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:15.313391+0000 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:15.313391+0000 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:15.309 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:15.310 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.312+0000 7fb6679bd640 1 Processor -- start 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 1 -- start start 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient: init 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient: _add_conns ranks=[8,6,4,3,2,5,0,1,7] 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient(hunting): picked mon.noname-e con 0x7fb660134910 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient(hunting): picked mon.noname-d con 0x7fb6600a0bc0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:15.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient(hunting): picked mon.noname-c con 0x7fb66013e340 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6600a2820 con 0x7fb66013e340 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6600a2b00 con 0x7fb6600a0bc0 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6600a1540 con 0x7fb660134910 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb6679bd640 10 monclient(hunting): authenticate will time out at 843.428317s 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb665759640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fb660134910 0x7fb660134cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54890/0 (socket says 172.21.15.7:54890) 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb665759640 1 -- 172.21.15.7:0/1300138960 learned_addr learned my addr 172.21.15.7:0/1300138960 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb665f5a640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fb66013e340 0x7fb660140730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:41064/0 (socket says 172.21.15.7:41064) 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.313+0000 7fb664f58640 1 --1- 172.21.15.7:0/1300138960 >> v1:172.21.15.7:6792/0 conn(0x7fb6600a0bc0 0x7fb66013cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56940/0 (socket says 172.21.15.7:56940) 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1405720463 0 0) 0x7fb6600a2b00 con 0x7fb6600a0bc0 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): my global_id is 74373 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb64c003520 con 0x7fb6600a0bc0 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 141011467 0 0) 0x7fb6600a1540 con 0x7fb660134910 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): my global_id is 4346 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6600a2b00 con 0x7fb660134910 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 602838475 0 0) 0x7fb6600a2820 con 0x7fb66013e340 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient(hunting): my global_id is 4342 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6600a1540 con 0x7fb66013e340 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4026581382 0 0) 0x7fb64c003520 con 0x7fb6600a0bc0 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 >> v1:172.21.15.7:6793/0 conn(0x7fb660134910 legacy=0x7fb660134cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 >> v1:172.21.15.7:6791/0 conn(0x7fb66013e340 legacy=0x7fb660140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 monclient: found mon.noname-d 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6600a3880 con 0x7fb6600a0bc0 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: _finish_auth 0 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb6500017b0 con 0x7fb6600a0bc0 2024-05-08T15:47:15.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb6679bd640 5 monclient: authenticate success, global_id 74373 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb6679bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: dump: 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.321 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb6679bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb650001ad0 con 0x7fb6600a0bc0 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb6679bd640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 1 -- 172.21.15.7:0/1300138960 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb650001d40 con 0x7fb6600a0bc0 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb6679bd640 4 set_mon_vals no callback set 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:15.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.314+0000 7fb657fff640 10 monclient: dump: 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.323 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 10 monclient: shutdown 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 1 -- 172.21.15.7:0/1300138960 >> v1:172.21.15.7:6792/0 conn(0x7fb6600a0bc0 legacy=0x7fb66013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 1 -- 172.21.15.7:0/1300138960 shutdown_connections 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 1 -- 172.21.15.7:0/1300138960 >> 172.21.15.7:0/1300138960 conn(0x7fb6600a9dc0 msgr2=0x7fb6600aa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 1 -- 172.21.15.7:0/1300138960 shutdown_connections 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 1 -- 172.21.15.7:0/1300138960 wait complete. 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 10 monclient: build_initial_monmap 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.315+0000 7fb6679bd640 10 monclient: monmap: 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:15.324 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 1 Processor -- start 2024-05-08T15:47:15.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 1 -- start start 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient: init 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient: _add_conns ranks=[6,4,3,7,5,1,0,8,2] 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient(hunting): picked mon.noname-d con 0x7fb6600a0bc0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient(hunting): picked mon.noname-c con 0x7fb660134910 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient(hunting): picked mon.noname-g con 0x7fb66013e340 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66013c790 con 0x7fb66013e340 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66009ba20 con 0x7fb660134910 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb66013d510 con 0x7fb6600a0bc0 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb6679bd640 10 monclient(hunting): authenticate will time out at 843.431147s 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb665759640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fb6600a0bc0 0x7fb66009ff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56950/0 (socket says 172.21.15.7:56950) 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb665759640 1 -- 172.21.15.7:0/2605031229 learned_addr learned my addr 172.21.15.7:0/2605031229 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:15.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 678719231 0 0) 0x7fb66013d510 con 0x7fb6600a0bc0 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): my global_id is 74382 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb644003520 con 0x7fb6600a0bc0 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3863442365 0 0) 0x7fb66009ba20 con 0x7fb660134910 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 10 monclient(hunting): my global_id is 4351 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.316+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb66013d510 con 0x7fb660134910 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 329850610 0 0) 0x7fb644003520 con 0x7fb6600a0bc0 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb66009ba20 con 0x7fb6600a0bc0 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1869654395 0 0) 0x7fb66013d510 con 0x7fb660134910 2024-05-08T15:47:15.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb644003520 con 0x7fb660134910 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb65c001550 con 0x7fb6600a0bc0 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.328 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb650001770 con 0x7fb660134910 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.329 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186038189 0 0) 0x7fb66013c790 con 0x7fb66013e340 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient(hunting): my global_id is 74388 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb66013d510 con 0x7fb66013e340 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1019055236 0 0) 0x7fb66009ba20 con 0x7fb6600a0bc0 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 >> v1:172.21.15.7:6791/0 conn(0x7fb660134910 legacy=0x7fb6601d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 >> v1:172.21.15.81:6790/0 conn(0x7fb66013e340 legacy=0x7fb6601d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 monclient: found mon.d 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6601c9ad0 con 0x7fb6600a0bc0 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:15.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6679bd640 5 monclient: authenticate success, global_id 74382 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6679bd640 10 monclient: _renew_subs 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6679bd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb6600a0690 con 0x7fb6600a0bc0 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb65c004a90 con 0x7fb6600a0bc0 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb65c004f80 con 0x7fb6600a0bc0 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb655ffb640 10 monclient: dump: 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.332 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6679bd640 10 monclient: _renew_subs 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6577fe640 4 set_mon_vals no callback set 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6679bd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.317+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fb6601c9250 con 0x7fb6600a0bc0 2024-05-08T15:47:15.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.318+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fb65c010300 con 0x7fb6600a0bc0 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.318+0000 7fb6679bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.318+0000 7fb6679bd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.318+0000 7fb6679bd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.318+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6600753d0 con 0x7fb6600a0bc0 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.318+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fb65c046ec0 con 0x7fb6600a0bc0 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.318+0000 7fb664f58640 10 monclient: get_auth_request con 0x7fb64403a0c0 auth_method 0 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.323+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fb65c016050 con 0x7fb6600a0bc0 2024-05-08T15:47:15.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.323+0000 7fb655ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.323+0000 7fb655ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.548+0000 7fb6679bd640 10 start_mon_command cmd=[{"prefix": "health", "format": "json"}] 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.548+0000 7fb6679bd640 10 monclient: _send_command 2 [{"prefix": "health", "format": "json"}] 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.548+0000 7fb6679bd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.548+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb66013fe30 con 0x7fb6600a0bc0 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.548+0000 7fb655ffb640 1 -- 172.21.15.7:0/2605031229 <== mon.6 v1:172.21.15.7:6792/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7fb65c015950 con 0x7fb6600a0bc0 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.548+0000 7fb655ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "health", "format": "json"}] 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.548+0000 7fb655ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:15.545 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:15.546 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T15:47:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 >> v1:172.21.15.81:6812/847627118 conn(0x7fb64403a0c0 legacy=0x7fb64403c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 10 monclient: shutdown 2024-05-08T15:47:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 >> v1:172.21.15.7:6792/0 conn(0x7fb6600a0bc0 legacy=0x7fb66009ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 shutdown_connections 2024-05-08T15:47:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 >> 172.21.15.7:0/2605031229 conn(0x7fb6600a9dc0 msgr2=0x7fb660137d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 shutdown_connections 2024-05-08T15:47:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.551+0000 7fb6679bd640 1 -- 172.21.15.7:0/2605031229 wait complete. 2024-05-08T15:47:15.558 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T15:47:15.558 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-05-08T15:47:15.571 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-05-08T15:47:15.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T15:47:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.678+0000 7fdaadb4b640 10 monclient: get_monmap_and_config 2024-05-08T15:47:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.678+0000 7fdaadb4b640 10 monclient: build_initial_monmap 2024-05-08T15:47:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.678+0000 7fdaadb4b640 10 monclient: monmap: 2024-05-08T15:47:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:15.674 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:15.679081+0000 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:15.679081+0000 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:15.675 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.678+0000 7fdaadb4b640 1 Processor -- start 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.678+0000 7fdaadb4b640 1 -- start start 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.678+0000 7fdaadb4b640 10 monclient: init 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 10 monclient: _add_conns ranks=[6,5,2,0,1,8,3,7,4] 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 10 monclient(hunting): picked mon.noname-d con 0x7fdaa80a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 10 monclient(hunting): picked mon.noname-h con 0x7fdaa80a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 10 monclient(hunting): picked mon.noname-b con 0x7fdaa813e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa80a1660 con 0x7fdaa813e310 2024-05-08T15:47:15.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa80a28f0 con 0x7fdaa80a5780 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa80a2bd0 con 0x7fdaa80a53a0 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaadb4b640 10 monclient(hunting): authenticate will time out at 843.794100s 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa77fe640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fdaa80a53a0 0x7fdaa80a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56960/0 (socket says 172.21.15.7:56960) 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa77fe640 1 -- 172.21.15.7:0/2532180508 learned_addr learned my addr 172.21.15.7:0/2532180508 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3236470689 0 0) 0x7fdaa80a2bd0 con 0x7fdaa80a53a0 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): my global_id is 74391 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda90003520 con 0x7fdaa80a53a0 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4148388081 0 0) 0x7fdaa80a1660 con 0x7fdaa813e310 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 10 monclient(hunting): my global_id is 74342 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.679+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaa80a2bd0 con 0x7fdaa813e310 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1059497826 0 0) 0x7fdaa80a28f0 con 0x7fdaa80a5780 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient(hunting): my global_id is 4397 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaa80a1660 con 0x7fdaa80a5780 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2126910923 0 0) 0x7fda90003520 con 0x7fdaa80a53a0 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 >> v1:172.21.15.81:6791/0 conn(0x7fdaa80a5780 legacy=0x7fdaa813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 >> v1:172.21.15.7:6790/0 conn(0x7fdaa813e310 legacy=0x7fdaa8140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 monclient: found mon.noname-d 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa80a6730 con 0x7fdaa80a53a0 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 5 monclient: authenticate success, global_id 74391 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fda980017b0 con 0x7fdaa80a53a0 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:15.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: dump: 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.686 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fda98001a10 con 0x7fdaa80a53a0 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 1 -- 172.21.15.7:0/2532180508 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fda98004be0 con 0x7fdaa80a53a0 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaa67fc640 10 monclient: dump: 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.687 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 4 set_mon_vals no callback set 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 10 monclient: shutdown 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 1 -- 172.21.15.7:0/2532180508 >> v1:172.21.15.7:6792/0 conn(0x7fdaa80a53a0 legacy=0x7fdaa80a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 1 -- 172.21.15.7:0/2532180508 shutdown_connections 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 1 -- 172.21.15.7:0/2532180508 >> 172.21.15.7:0/2532180508 conn(0x7fdaa80a9fa0 msgr2=0x7fdaa80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 1 -- 172.21.15.7:0/2532180508 shutdown_connections 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.680+0000 7fdaadb4b640 1 -- 172.21.15.7:0/2532180508 wait complete. 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.681+0000 7fdaadb4b640 10 monclient: build_initial_monmap 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.681+0000 7fdaadb4b640 10 monclient: monmap: 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:15.689 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.681+0000 7fdaadb4b640 1 Processor -- start 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.681+0000 7fdaadb4b640 1 -- start start 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.681+0000 7fdaadb4b640 10 monclient: init 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 10 monclient: _add_conns ranks=[5,3,2,8,7,4,6,0,1] 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 10 monclient(hunting): picked mon.noname-h con 0x7fdaa80a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:15.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 10 monclient(hunting): picked mon.noname-g con 0x7fdaa80a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 10 monclient(hunting): picked mon.noname-b con 0x7fdaa813e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa81cd1d0 con 0x7fdaa813e310 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa81357b0 con 0x7fdaa80a5780 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa8137200 con 0x7fdaa80a53a0 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaadb4b640 10 monclient(hunting): authenticate will time out at 843.797029s 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaa7fff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fdaa813e310 0x7fdaa81d4730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:32904/0 (socket says 172.21.15.7:32904) 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fdaa7fff640 1 -- 172.21.15.7:0/674493819 learned_addr learned my addr 172.21.15.7:0/674493819 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014837071 0 0) 0x7fdaa81cd1d0 con 0x7fdaa813e310 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): my global_id is 74351 2024-05-08T15:47:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda94003520 con 0x7fdaa813e310 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 717105898 0 0) 0x7fda94003520 con 0x7fdaa813e310 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdaa81cd1d0 con 0x7fdaa813e310 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdaa0001d60 con 0x7fdaa813e310 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.682+0000 7fda8ffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4079135475 0 0) 0x7fdaa8137200 con 0x7fdaa80a53a0 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): my global_id is 4406 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda94003520 con 0x7fdaa80a53a0 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009629968 0 0) 0x7fdaa81357b0 con 0x7fdaa80a5780 2024-05-08T15:47:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient(hunting): my global_id is 74397 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaa8137200 con 0x7fdaa80a5780 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2327059667 0 0) 0x7fdaa81cd1d0 con 0x7fdaa813e310 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 >> v1:172.21.15.81:6791/0 conn(0x7fdaa80a53a0 legacy=0x7fdaa81cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 >> v1:172.21.15.81:6790/0 conn(0x7fdaa80a5780 legacy=0x7fdaa81ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 monclient: found mon.b 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa8136580 con 0x7fdaa813e310 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: _finish_auth 0 2024-05-08T15:47:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaadb4b640 5 monclient: authenticate success, global_id 74351 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaadb4b640 10 monclient: _renew_subs 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaadb4b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa81384b0 con 0x7fdaa813e310 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaadb4b640 10 monclient: _renew_subs 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaadb4b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fdaa81d6030 con 0x7fdaa813e310 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdaa0002080 con 0x7fdaa813e310 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdaa0003fe0 con 0x7fdaa813e310 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fdaa5ffb640 4 set_mon_vals no callback set 2024-05-08T15:47:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.683+0000 7fda8ffff640 10 monclient: dump: 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:15.696 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.684+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fdaa0004180 con 0x7fdaa813e310 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.684+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fdaa0046f60 con 0x7fdaa813e310 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.684+0000 7fdaadb4b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.684+0000 7fdaadb4b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.684+0000 7fdaadb4b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.684+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda88005080 con 0x7fdaa813e310 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.684+0000 7fdaa77fe640 10 monclient: get_auth_request con 0x7fda94039c90 auth_method 0 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.689+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fdaa0016150 con 0x7fdaa813e310 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.689+0000 7fda8ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.689+0000 7fda8ffff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.871+0000 7fdaadb4b640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.871+0000 7fdaadb4b640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.871+0000 7fdaadb4b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.871+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fda88005370 con 0x7fdaa813e310 2024-05-08T15:47:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.872+0000 7fda8ffff640 1 -- 172.21.15.7:0/674493819 <== mon.2 v1:172.21.15.7:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+10735 (unknown 3338201251 0 1285910653) 0x7fdaa0015a50 con 0x7fdaa813e310 2024-05-08T15:47:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.872+0000 7fda8ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:47:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.872+0000 7fda8ffff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:15.868 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:15.869 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":18,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:47:12.166909+0000","last_up_change":"2024-05-08T15:47:03.939015+0000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T15:47:03.139226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T15:47:09.761977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6804","nonce":3231689356}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6805","nonce":3231689356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6807","nonce":3231689356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6806","nonce":3231689356}]},"public_addr":"172.21.15.7:6804/3231689356","cluster_addr":"172.21.15.7:6805/3231689356","heartbeat_back_addr":"172.21.15.7:6807/3231689356","heartbeat_front_addr":"172.21.15.7:6806/3231689356","state":["exists","up"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6800","nonce":601836183}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6801","nonce":601836183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6803","nonce":601836183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6802","nonce":601836183}]},"public_addr":"172.21.15.7:6800/601836183","cluster_addr":"172.21.15.7:6801/601836183","heartbeat_back_addr":"172.21.15.7:6803/601836183","heartbeat_front_addr":"172.21.15.7:6802/601836183","state":["exists","up"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6808","nonce":2598303896}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6809","nonce":2598303896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6811","nonce":2598303896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6810","nonce":2598303896}]},"public_addr":"172.21.15.7:6808/2598303896","cluster_addr":"172.21.15.7:6809/2598303896","heartbeat_back_addr":"172.21.15.7:6811/2598303896","heartbeat_front_addr":"172.21.15.7:6810/2598303896","state":["exists","up"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6802","nonce":1326537731}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6805","nonce":1326537731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6807","nonce":1326537731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6806","nonce":1326537731}]},"public_addr":"172.21.15.81:6802/1326537731","cluster_addr":"172.21.15.81:6805/1326537731","heartbeat_back_addr":"172.21.15.81:6807/1326537731","heartbeat_front_addr":"172.21.15.81:6806/1326537731","state":["exists","up"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6808","nonce":193838907}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6809","nonce":193838907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6811","nonce":193838907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6810","nonce":193838907}]},"public_addr":"172.21.15.81:6808/193838907","cluster_addr":"172.21.15.81:6809/193838907","heartbeat_back_addr":"172.21.15.81:6811/193838907","heartbeat_front_addr":"172.21.15.81:6810/193838907","state":["exists","up"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6800","nonce":385867939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6801","nonce":385867939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6804","nonce":385867939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6803","nonce":385867939}]},"public_addr":"172.21.15.81:6800/385867939","cluster_addr":"172.21.15.81:6801/385867939","heartbeat_back_addr":"172.21.15.81:6804/385867939","heartbeat_front_addr":"172.21.15.81:6803/385867939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.702217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.485746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.746567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.749309+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.745027+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.779789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 >> v1:172.21.15.81:6812/847627118 conn(0x7fda94039c90 legacy=0x7fda9403c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 10 monclient: shutdown 2024-05-08T15:47:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 >> v1:172.21.15.7:6790/0 conn(0x7fdaa813e310 legacy=0x7fdaa81d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 shutdown_connections 2024-05-08T15:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 >> 172.21.15.7:0/674493819 conn(0x7fdaa80a9fa0 msgr2=0x7fdaa8140c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 shutdown_connections 2024-05-08T15:47:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.874+0000 7fdaadb4b640 1 -- 172.21.15.7:0/674493819 wait complete. 2024-05-08T15:47:15.881 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T15:47:03.139226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T15:47:09.761977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T15:47:15.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.992+0000 7f9b10fa4640 10 monclient: get_monmap_and_config 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.992+0000 7f9b10fa4640 10 monclient: build_initial_monmap 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.992+0000 7f9b10fa4640 10 monclient: monmap: 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:15.993825+0000 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:15.993825+0000 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:15.989 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:15.990 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.993+0000 7f9b10fa4640 1 Processor -- start 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.993+0000 7f9b10fa4640 1 -- start start 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.993+0000 7f9b10fa4640 10 monclient: init 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient: _add_conns ranks=[3,0,8,7,6,2,1,5,4] 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient(hunting): picked mon.noname-g con 0x7f9b0c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient(hunting): picked mon.noname-a con 0x7f9b0c0a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient(hunting): picked mon.noname-e con 0x7f9b0c0a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c0a2c70 con 0x7f9b0c0a49a0 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c0a2f50 con 0x7f9b0c137940 2024-05-08T15:47:15.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c0a1670 con 0x7f9b0c0a41b0 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient(hunting): authenticate will time out at 844.108781s 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0ad9d640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f9b0c0a41b0 0x7f9b0c140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54896/0 (socket says 172.21.15.7:54896) 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0ad9d640 1 -- 172.21.15.7:0/4118489031 learned_addr learned my addr 172.21.15.7:0/4118489031 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2084896215 0 0) 0x7f9b0c0a2c70 con 0x7f9b0c0a49a0 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): my global_id is 4419 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9af8003520 con 0x7f9b0c0a49a0 2024-05-08T15:47:15.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2381722898 0 0) 0x7f9b0c0a1670 con 0x7f9b0c0a41b0 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): my global_id is 4355 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b0c0a2c70 con 0x7f9b0c0a41b0 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1201790651 0 0) 0x7f9b0c0a2f50 con 0x7f9b0c137940 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient(hunting): my global_id is 74406 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b0c0a1670 con 0x7f9b0c137940 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2581330185 0 0) 0x7f9af8003520 con 0x7f9b0c0a49a0 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 >> v1:172.21.15.7:6793/0 conn(0x7f9b0c0a41b0 legacy=0x7f9b0c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:15.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 >> v1:172.21.15.81:6790/0 conn(0x7f9b0c137940 legacy=0x7f9b0c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 monclient: found mon.noname-a 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c0a39f0 con 0x7f9b0c0a49a0 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9af40017b0 con 0x7f9b0c0a49a0 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 5 monclient: authenticate success, global_id 4419 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: dump: 2024-05-08T15:47:16.000 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9af4001ad0 con 0x7f9b0c0a49a0 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 1 -- 172.21.15.7:0/4118489031 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9af4001d40 con 0x7f9b0c0a49a0 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b10fa4640 4 set_mon_vals no callback set 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.994+0000 7f9b0959a640 10 monclient: dump: 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.002 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.995+0000 7f9b10fa4640 10 monclient: shutdown 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.995+0000 7f9b10fa4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.995+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4118489031 >> v1:172.21.15.7:6789/0 conn(0x7f9b0c0a49a0 legacy=0x7f9b0c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.995+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4118489031 shutdown_connections 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.995+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4118489031 >> 172.21.15.7:0/4118489031 conn(0x7f9b0c0a9fa0 msgr2=0x7f9b0c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:16.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.995+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4118489031 shutdown_connections 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.995+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4118489031 wait complete. 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.996+0000 7f9b10fa4640 10 monclient: build_initial_monmap 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.996+0000 7f9b10fa4640 10 monclient: monmap: 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.996+0000 7f9b10fa4640 1 Processor -- start 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.996+0000 7f9b10fa4640 1 -- start start 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.996+0000 7f9b10fa4640 10 monclient: init 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 10 monclient: _add_conns ranks=[4,1,2,0,3,6,7,8,5] 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 10 monclient(hunting): picked mon.noname-c con 0x7f9b0c0a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 10 monclient(hunting): picked mon.noname-f con 0x7f9b0c0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 10 monclient(hunting): picked mon.noname-b con 0x7f9b0c137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c134210 con 0x7f9b0c0a49a0 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c0a1f30 con 0x7f9b0c137940 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b0c136030 con 0x7f9b0c0a41b0 2024-05-08T15:47:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b10fa4640 10 monclient(hunting): authenticate will time out at 844.112015s 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b0a59c640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f9b0c0a41b0 0x7f9b0c1ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:41090/0 (socket says 172.21.15.7:41090) 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b0a59c640 1 -- 172.21.15.7:0/4061719134 learned_addr learned my addr 172.21.15.7:0/4061719134 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3303616654 0 0) 0x7f9b04001a10 con 0x7f9b0c137940 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): my global_id is 74360 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9aec003520 con 0x7f9b0c137940 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804566610 0 0) 0x7f9b0c136030 con 0x7f9b0c0a41b0 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): my global_id is 4360 2024-05-08T15:47:16.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b04001a10 con 0x7f9b0c0a41b0 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 248873403 0 0) 0x7f9aec003520 con 0x7f9b0c137940 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9b0c136030 con 0x7f9b0c137940 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355156208 0 0) 0x7f9b0c134210 con 0x7f9b0c0a49a0 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): my global_id is 4384 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9aec003520 con 0x7f9b0c0a49a0 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9b04001c60 con 0x7f9b0c137940 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.007 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.997+0000 7f9b02ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2289138191 0 0) 0x7f9b04001a10 con 0x7f9b0c0a41b0 2024-05-08T15:47:16.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9b0c134210 con 0x7f9b0c0a41b0 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9afc001d90 con 0x7f9b0c0a41b0 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.009 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1035141640 0 0) 0x7f9b0c136030 con 0x7f9b0c137940 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 >> v1:172.21.15.7:6791/0 conn(0x7f9b0c0a41b0 legacy=0x7f9b0c1ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 >> v1:172.21.15.81:6789/0 conn(0x7f9b0c0a49a0 legacy=0x7f9b0c1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 monclient: found mon.b 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c1cda40 con 0x7f9b0c137940 2024-05-08T15:47:16.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b10fa4640 5 monclient: authenticate success, global_id 74360 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b10fa4640 10 monclient: _renew_subs 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b10fa4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c0a0f60 con 0x7f9b0c137940 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b10fa4640 10 monclient: _renew_subs 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b10fa4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9b0c135d50 con 0x7f9b0c137940 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9b04004a90 con 0x7f9b0c137940 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9b04004f80 con 0x7f9b0c137940 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b08d99640 4 set_mon_vals no callback set 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 10 monclient: dump: 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.012 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.998+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f9b04010300 con 0x7f9b0c137940 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.999+0000 7f9b10fa4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.999+0000 7f9b10fa4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.999+0000 7f9b10fa4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.999+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af0005080 con 0x7f9b0c137940 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.999+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f9b04046e90 con 0x7f9b0c137940 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:15.999+0000 7f9b0a59c640 10 monclient: get_auth_request con 0x7f9aec03a210 auth_method 0 2024-05-08T15:47:16.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.004+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f9b0c0a1f30 con 0x7f9b0c137940 2024-05-08T15:47:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.004+0000 7f9b02ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.004+0000 7f9b02ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.187+0000 7f9b10fa4640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-05-08T15:47:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.187+0000 7f9b10fa4640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-05-08T15:47:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.187+0000 7f9b10fa4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.187+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9af0005b80 con 0x7f9b0c137940 2024-05-08T15:47:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.189+0000 7f9b02ffd640 1 -- 172.21.15.7:0/4061719134 <== mon.2 v1:172.21.15.7:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) ==== 93+0+10 (unknown 4068545244 0 2170607528) 0x7f9b04016080 con 0x7f9b0c137940 2024-05-08T15:47:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.189+0000 7f9b02ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}] 2024-05-08T15:47:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.189+0000 7f9b02ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:16.186 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-05-08T15:47:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.191+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 >> v1:172.21.15.81:6812/847627118 conn(0x7f9aec03a210 legacy=0x7f9aec03c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.191+0000 7f9b10fa4640 10 monclient: shutdown 2024-05-08T15:47:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.191+0000 7f9b10fa4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.191+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 >> v1:172.21.15.7:6790/0 conn(0x7f9b0c137940 legacy=0x7f9b0c1d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.191+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 shutdown_connections 2024-05-08T15:47:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.191+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 >> 172.21.15.7:0/4061719134 conn(0x7f9b0c0a9fa0 msgr2=0x7f9b0c130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.192+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 shutdown_connections 2024-05-08T15:47:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.192+0000 7f9b10fa4640 1 -- 172.21.15.7:0/4061719134 wait complete. 2024-05-08T15:47:16.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.307+0000 7f621cd85640 10 monclient: get_monmap_and_config 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.307+0000 7f621cd85640 10 monclient: build_initial_monmap 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.307+0000 7f621cd85640 10 monclient: monmap: 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:16.308831+0000 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:16.308831+0000 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:16.304 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:16.305 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.308+0000 7f621cd85640 1 Processor -- start 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.308+0000 7f621cd85640 1 -- start start 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.308+0000 7f621cd85640 10 monclient: init 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient: _add_conns ranks=[5,0,7,6,4,2,1,8,3] 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient(hunting): picked mon.noname-h con 0x7f6218137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient(hunting): picked mon.noname-a con 0x7f62180a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient(hunting): picked mon.noname-i con 0x7f62180a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62180a2c70 con 0x7f62180a49a0 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62180a2f50 con 0x7f6218137940 2024-05-08T15:47:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62180a1670 con 0x7f62180a41b0 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient(hunting): authenticate will time out at 844.423774s 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f6215d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f62180a49a0 0x7f621813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58126/0 (socket says 172.21.15.7:58126) 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f6215d9b640 1 -- 172.21.15.7:0/2211934268 learned_addr learned my addr 172.21.15.7:0/2211934268 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3801139975 0 0) 0x7f62180a2c70 con 0x7f62180a49a0 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): my global_id is 4428 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6200003520 con 0x7f62180a49a0 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205425969 0 0) 0x7f62180a2f50 con 0x7f6218137940 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): my global_id is 4415 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62180a2c70 con 0x7f6218137940 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567138841 0 0) 0x7f62180a1670 con 0x7f62180a41b0 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient(hunting): my global_id is 4390 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62180a2f50 con 0x7f62180a41b0 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 182454968 0 0) 0x7f6200003520 con 0x7f62180a49a0 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 >> v1:172.21.15.81:6792/0 conn(0x7f62180a41b0 legacy=0x7f6218140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 >> v1:172.21.15.81:6791/0 conn(0x7f6218137940 legacy=0x7f6218137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 monclient: found mon.noname-a 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62180a39f0 con 0x7f62180a49a0 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f620c0017b0 con 0x7f62180a49a0 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 5 monclient: authenticate success, global_id 4428 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: dump: 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f620c001ad0 con 0x7f62180a49a0 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 1 -- 172.21.15.7:0/2211934268 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f620c004be0 con 0x7f62180a49a0 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621cd85640 4 set_mon_vals no callback set 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.309+0000 7f621559a640 10 monclient: dump: 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.310+0000 7f621cd85640 10 monclient: shutdown 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.310+0000 7f621cd85640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.310+0000 7f621cd85640 1 -- 172.21.15.7:0/2211934268 >> v1:172.21.15.7:6789/0 conn(0x7f62180a49a0 legacy=0x7f621813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.310+0000 7f621cd85640 1 -- 172.21.15.7:0/2211934268 shutdown_connections 2024-05-08T15:47:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.310+0000 7f621cd85640 1 -- 172.21.15.7:0/2211934268 >> 172.21.15.7:0/2211934268 conn(0x7f62180a9fa0 msgr2=0x7f62180aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.310+0000 7f621cd85640 1 -- 172.21.15.7:0/2211934268 shutdown_connections 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.310+0000 7f621cd85640 1 -- 172.21.15.7:0/2211934268 wait complete. 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.311+0000 7f621cd85640 10 monclient: build_initial_monmap 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.311+0000 7f621cd85640 10 monclient: monmap: 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.311+0000 7f621cd85640 1 Processor -- start 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.311+0000 7f621cd85640 1 -- start start 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.311+0000 7f621cd85640 10 monclient: init 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 10 monclient: _add_conns ranks=[8,6,0,1,4,7,2,5,3] 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 10 monclient(hunting): picked mon.noname-e con 0x7f62180a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 10 monclient(hunting): picked mon.noname-d con 0x7f62180a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 10 monclient(hunting): picked mon.noname-a con 0x7f6218137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62181332b0 con 0x7f6218137940 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6218135850 con 0x7f62180a49a0 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6218135e10 con 0x7f62180a41b0 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621cd85640 10 monclient(hunting): authenticate will time out at 844.427061s 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621659c640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f62180a41b0 0x7f6218136510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54902/0 (socket says 172.21.15.7:54902) 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f621659c640 1 -- 172.21.15.7:0/1482454681 learned_addr learned my addr 172.21.15.7:0/1482454681 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6215d9b640 1 --1- 172.21.15.7:0/1482454681 >> v1:172.21.15.7:6792/0 conn(0x7f62180a49a0 0x7f6218132490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56966/0 (socket says 172.21.15.7:56966) 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1645279576 0 0) 0x7f6218135e10 con 0x7f62180a41b0 2024-05-08T15:47:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): my global_id is 4364 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61f8003520 con 0x7f62180a41b0 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3959463852 0 0) 0x7f62181332b0 con 0x7f6218137940 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): my global_id is 4437 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6218135e10 con 0x7f6218137940 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3934653528 0 0) 0x7f6218135850 con 0x7f62180a49a0 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 10 monclient(hunting): my global_id is 74400 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.312+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62181332b0 con 0x7f62180a49a0 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 688308528 0 0) 0x7f61f8003520 con 0x7f62180a41b0 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6218135850 con 0x7f62180a41b0 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f62080019f0 con 0x7f62180a41b0 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1086024047 0 0) 0x7f6218135e10 con 0x7f6218137940 2024-05-08T15:47:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f61f8003520 con 0x7f6218137940 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6210001dd0 con 0x7f6218137940 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3700601690 0 0) 0x7f62181332b0 con 0x7f62180a49a0 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6218135e10 con 0x7f62180a49a0 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f620c004b20 con 0x7f62180a49a0 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3220904367 0 0) 0x7f6218135850 con 0x7f62180a41b0 2024-05-08T15:47:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 >> v1:172.21.15.7:6792/0 conn(0x7f62180a49a0 legacy=0x7f6218132490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 >> v1:172.21.15.7:6789/0 conn(0x7f6218137940 legacy=0x7f6218132ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 monclient: found mon.e 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62181360f0 con 0x7f62180a41b0 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f621cd85640 5 monclient: authenticate success, global_id 4364 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f621cd85640 10 monclient: _renew_subs 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f621cd85640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6218135b30 con 0x7f62180a41b0 2024-05-08T15:47:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f621cd85640 10 monclient: _renew_subs 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f621cd85640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f62181d5e90 con 0x7f62180a41b0 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f62080038c0 con 0x7f62180a41b0 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6208003e30 con 0x7f62180a41b0 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6206ffd640 10 monclient: dump: 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.313+0000 7f6214d99640 4 set_mon_vals no callback set 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.314+0000 7f621cd85640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.314+0000 7f621cd85640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.314+0000 7f621cd85640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.314+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62180753d0 con 0x7f62180a41b0 2024-05-08T15:47:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.314+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6208004050 con 0x7f62180a41b0 2024-05-08T15:47:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.315+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f6208046dd0 con 0x7f62180a41b0 2024-05-08T15:47:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.315+0000 7f6215d9b640 10 monclient: get_auth_request con 0x7f61f803a310 auth_method 0 2024-05-08T15:47:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.319+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f6208015ea0 con 0x7f62180a41b0 2024-05-08T15:47:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.319+0000 7f6206ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.319+0000 7f6206ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.502+0000 7f621cd85640 10 start_mon_command cmd=[{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.502+0000 7f621cd85640 10 monclient: _send_command 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.502+0000 7f621cd85640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.502+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) -- 0x7f6218141c20 con 0x7f62180a41b0 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.502+0000 7f6206ffd640 1 -- 172.21.15.7:0/1482454681 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) ==== 92+0+10 (unknown 1614612280 0 265718887) 0x7f6208004440 con 0x7f62180a41b0 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.502+0000 7f6206ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}] 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.502+0000 7f6206ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:16.499 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 8 2024-05-08T15:47:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.504+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 >> v1:172.21.15.81:6812/847627118 conn(0x7f61f803a310 legacy=0x7f61f803c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.504+0000 7f621cd85640 10 monclient: shutdown 2024-05-08T15:47:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.504+0000 7f621cd85640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.504+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 >> v1:172.21.15.7:6793/0 conn(0x7f62180a41b0 legacy=0x7f6218136510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.504+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 shutdown_connections 2024-05-08T15:47:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.504+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 >> 172.21.15.7:0/1482454681 conn(0x7f62180a9fa0 msgr2=0x7f6218130c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:16.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.505+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 shutdown_connections 2024-05-08T15:47:16.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.505+0000 7f621cd85640 1 -- 172.21.15.7:0/1482454681 wait complete. 2024-05-08T15:47:16.512 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-05-08T15:47:16.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:47:16.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.618+0000 7fca88cb3640 10 monclient: get_monmap_and_config 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.618+0000 7fca88cb3640 10 monclient: build_initial_monmap 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.618+0000 7fca88cb3640 10 monclient: monmap: 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:16.619461+0000 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:16.619461+0000 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:16.615 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:16.616 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 1 Processor -- start 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 1 -- start start 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient: init 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient: _add_conns ranks=[6,8,1,5,7,4,3,0,2] 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient(hunting): picked mon.noname-d con 0x7fca841359f0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient(hunting): picked mon.noname-e con 0x7fca841354c0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient(hunting): picked mon.noname-f con 0x7fca8413bf20 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca84072510 con 0x7fca8413bf20 2024-05-08T15:47:16.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca840727f0 con 0x7fca841359f0 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca84072ad0 con 0x7fca841354c0 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca88cb3640 10 monclient(hunting): authenticate will time out at 844.734410s 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca81d9b640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fca841354c0 0x7fca8413a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54904/0 (socket says 172.21.15.7:54904) 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca8259c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fca841359f0 0x7fca84135dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56968/0 (socket says 172.21.15.7:56968) 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.619+0000 7fca81d9b640 1 -- 172.21.15.7:0/4079556253 learned_addr learned my addr 172.21.15.7:0/4079556253 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389989656 0 0) 0x7fca840727f0 con 0x7fca841359f0 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): my global_id is 74409 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca70003520 con 0x7fca841359f0 2024-05-08T15:47:16.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2584155881 0 0) 0x7fca84072ad0 con 0x7fca841354c0 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): my global_id is 4373 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca840727f0 con 0x7fca841354c0 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1256876323 0 0) 0x7fca84072510 con 0x7fca8413bf20 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient(hunting): my global_id is 4393 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca84072ad0 con 0x7fca8413bf20 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3937004892 0 0) 0x7fca70003520 con 0x7fca841359f0 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 >> v1:172.21.15.7:6793/0 conn(0x7fca841354c0 legacy=0x7fca8413a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 >> v1:172.21.15.81:6789/0 conn(0x7fca8413bf20 legacy=0x7fca8413e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 monclient: found mon.noname-d 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca840a3120 con 0x7fca841359f0 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca6c0016e0 con 0x7fca841359f0 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca88cb3640 5 monclient: authenticate success, global_id 74409 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca88cb3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:16.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: dump: 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.630 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca88cb3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fca6c001940 con 0x7fca841359f0 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 1 -- 172.21.15.7:0/4079556253 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca6c004b20 con 0x7fca841359f0 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca88cb3640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca88cb3640 4 set_mon_vals no callback set 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.620+0000 7fca8159a640 10 monclient: dump: 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.631 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.632 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 10 monclient: shutdown 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 1 -- 172.21.15.7:0/4079556253 >> v1:172.21.15.7:6792/0 conn(0x7fca841359f0 legacy=0x7fca84135dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 1 -- 172.21.15.7:0/4079556253 shutdown_connections 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 1 -- 172.21.15.7:0/4079556253 >> 172.21.15.7:0/4079556253 conn(0x7fca840a6280 msgr2=0x7fca840a2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 1 -- 172.21.15.7:0/4079556253 shutdown_connections 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 1 -- 172.21.15.7:0/4079556253 wait complete. 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 10 monclient: build_initial_monmap 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.621+0000 7fca88cb3640 10 monclient: monmap: 2024-05-08T15:47:16.633 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:16.634 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 1 Processor -- start 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 1 -- start start 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient: init 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient: _add_conns ranks=[6,8,4,0,2,5,3,1,7] 2024-05-08T15:47:16.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient(hunting): picked mon.noname-d con 0x7fca841354c0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient(hunting): picked mon.noname-e con 0x7fca841359f0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient(hunting): picked mon.noname-c con 0x7fca8413bf20 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca84130420 con 0x7fca8413bf20 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca840707d0 con 0x7fca841354c0 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca8409fa20 con 0x7fca841359f0 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca88cb3640 10 monclient(hunting): authenticate will time out at 844.737398s 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca8259c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fca841354c0 0x7fca8406f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56970/0 (socket says 172.21.15.7:56970) 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.622+0000 7fca8259c640 1 -- 172.21.15.7:0/1967062617 learned_addr learned my addr 172.21.15.7:0/1967062617 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4056967842 0 0) 0x7fca8409fa20 con 0x7fca841359f0 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): my global_id is 4382 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca64003520 con 0x7fca841359f0 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 247647143 0 0) 0x7fca840707d0 con 0x7fca841354c0 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): my global_id is 74418 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca8409fa20 con 0x7fca841354c0 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026352591 0 0) 0x7fca84130420 con 0x7fca8413bf20 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): my global_id is 4369 2024-05-08T15:47:16.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca840707d0 con 0x7fca8413bf20 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2670500671 0 0) 0x7fca64003520 con 0x7fca841359f0 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca84130420 con 0x7fca841359f0 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca74001510 con 0x7fca841359f0 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.638 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3559727476 0 0) 0x7fca8409fa20 con 0x7fca841354c0 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca64003520 con 0x7fca841354c0 2024-05-08T15:47:16.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca6c004b60 con 0x7fca841354c0 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.640 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2257249223 0 0) 0x7fca840707d0 con 0x7fca8413bf20 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca8409fa20 con 0x7fca8413bf20 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca7c001cb0 con 0x7fca8413bf20 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient(hunting): dump: 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.641 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.642 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3978380375 0 0) 0x7fca84130420 con 0x7fca841359f0 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 >> v1:172.21.15.7:6792/0 conn(0x7fca841354c0 legacy=0x7fca8406f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 >> v1:172.21.15.7:6791/0 conn(0x7fca8413bf20 legacy=0x7fca841d4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 monclient: found mon.e 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca84130140 con 0x7fca841359f0 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca88cb3640 5 monclient: authenticate success, global_id 4382 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca88cb3640 10 monclient: _renew_subs 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca88cb3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fca84070450 con 0x7fca841359f0 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca88cb3640 10 monclient: _renew_subs 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca88cb3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fca8413a9c0 con 0x7fca841359f0 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fca74004a90 con 0x7fca841359f0 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca74004f80 con 0x7fca841359f0 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7bfff640 4 set_mon_vals no callback set 2024-05-08T15:47:16.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.623+0000 7fca7affd640 10 monclient: dump: 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.646 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.647 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.648 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.624+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fca74010300 con 0x7fca841359f0 2024-05-08T15:47:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.624+0000 7fca88cb3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.624+0000 7fca88cb3640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.624+0000 7fca88cb3640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:16.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.624+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca68005080 con 0x7fca841359f0 2024-05-08T15:47:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.624+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fca74046cb0 con 0x7fca841359f0 2024-05-08T15:47:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.625+0000 7fca8259c640 10 monclient: get_auth_request con 0x7fca6403a320 auth_method 0 2024-05-08T15:47:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.629+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fca74015e40 con 0x7fca841359f0 2024-05-08T15:47:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.629+0000 7fca7affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.629+0000 7fca7affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:16.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.808+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fca68000b90 con 0x7fca6403a320 2024-05-08T15:47:16.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.810+0000 7fca7affd640 1 -- 172.21.15.7:0/1967062617 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33864 (unknown 2965378022 0 2814674891) 0x7fca68000b90 con 0x7fca6403a320 2024-05-08T15:47:16.807 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:16.807 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:47:16.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 >> v1:172.21.15.81:6812/847627118 conn(0x7fca6403a320 legacy=0x7fca6403c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 10 monclient: shutdown 2024-05-08T15:47:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 >> v1:172.21.15.7:6793/0 conn(0x7fca841359f0 legacy=0x7fca8406fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 shutdown_connections 2024-05-08T15:47:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 >> 172.21.15.7:0/1967062617 conn(0x7fca840a6280 msgr2=0x7fca840755e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 shutdown_connections 2024-05-08T15:47:16.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.812+0000 7fca88cb3640 1 -- 172.21.15.7:0/1967062617 wait complete. 2024-05-08T15:47:16.819 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-05-08T15:47:15.098479+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167268,"kb_used_data":976,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566063772,"statfs":{"total":579820584960,"available":579649302528,"internally_reserved":0,"allocated":999424,"data_stored":1324460,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":170187426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172443+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:47:12.172443+0000","last_peered":"2024-05-08T15:47:12.172443+0000","last_clean":"2024-05-08T15:47:12.172443+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:47:12.172443+0000","last_undegraded":"2024-05-08T15:47:12.172443+0000","last_fullsized":"2024-05-08T15:47:12.172443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:14.941756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172367+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:47:12.172367+0000","last_peered":"2024-05-08T15:47:12.172367+0000","last_clean":"2024-05-08T15:47:12.172367+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:47:12.172367+0000","last_undegraded":"2024-05-08T15:47:12.172367+0000","last_fullsized":"2024-05-08T15:47:12.172367+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:28.945664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.173944+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:30.537193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171573+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:47:12.171573+0000","last_peered":"2024-05-08T15:47:12.171573+0000","last_clean":"2024-05-08T15:47:12.171573+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:47:12.171573+0000","last_undegraded":"2024-05-08T15:47:12.171573+0000","last_fullsized":"2024-05-08T15:47:12.171573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171867+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:47:12.171867+0000","last_peered":"2024-05-08T15:47:12.171867+0000","last_clean":"2024-05-08T15:47:12.171867+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:47:12.171867+0000","last_undegraded":"2024-05-08T15:47:12.171867+0000","last_fullsized":"2024-05-08T15:47:12.171867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171427+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:47:12.171427+0000","last_peered":"2024-05-08T15:47:12.171427+0000","last_clean":"2024-05-08T15:47:12.171427+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:47:12.171427+0000","last_undegraded":"2024-05-08T15:47:12.171427+0000","last_fullsized":"2024-05-08T15:47:12.171427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:49.677393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171045+0000","last_change":"2024-05-08T15:47:11.176829+0000","last_active":"2024-05-08T15:47:12.171045+0000","last_peered":"2024-05-08T15:47:12.171045+0000","last_clean":"2024-05-08T15:47:12.171045+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:47:12.171045+0000","last_undegraded":"2024-05-08T15:47:12.171045+0000","last_fullsized":"2024-05-08T15:47:12.171045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:32.476618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:30.243149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172345+0000","last_change":"2024-05-08T15:47:04.948326+0000","last_active":"2024-05-08T15:47:12.172345+0000","last_peered":"2024-05-08T15:47:12.172345+0000","last_clean":"2024-05-08T15:47:12.172345+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:47:12.172345+0000","last_undegraded":"2024-05-08T15:47:12.172345+0000","last_fullsized":"2024-05-08T15:47:12.172345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_clean_scrub_stamp":"2024-05-08T15:47:03.939015+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:25.024655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:47:16.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:47:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.928+0000 7fbe3b932640 10 monclient: get_monmap_and_config 2024-05-08T15:47:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.928+0000 7fbe3b932640 10 monclient: build_initial_monmap 2024-05-08T15:47:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.928+0000 7fbe3b932640 10 monclient: monmap: 2024-05-08T15:47:16.924 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:16.929241+0000 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:16.929241+0000 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:16.925 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:16.926 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:16.926 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.928+0000 7fbe3b932640 1 Processor -- start 2024-05-08T15:47:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 1 -- start start 2024-05-08T15:47:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient: init 2024-05-08T15:47:16.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient: _add_conns ranks=[5,1,2,6,8,7,0,3,4] 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient(hunting): picked mon.noname-h con 0x7fbe34137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient(hunting): picked mon.noname-f con 0x7fbe340a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient(hunting): picked mon.noname-b con 0x7fbe340a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe340a2c70 con 0x7fbe340a49a0 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe340a2f50 con 0x7fbe340a41b0 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe340a1670 con 0x7fbe34137940 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe3b932640 10 monclient(hunting): authenticate will time out at 845.044203s 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe39ecf640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fbe340a41b0 0x7fbe34140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:32914/0 (socket says 172.21.15.7:32914) 2024-05-08T15:47:16.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe39ecf640 1 -- 172.21.15.7:0/1437851280 learned_addr learned my addr 172.21.15.7:0/1437851280 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 636214021 0 0) 0x7fbe340a2f50 con 0x7fbe340a41b0 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe2bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe2bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe2bfff640 10 monclient(hunting): my global_id is 74369 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.929+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe20003520 con 0x7fbe340a41b0 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2611750796 0 0) 0x7fbe340a2c70 con 0x7fbe340a49a0 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): my global_id is 4402 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe340a2f50 con 0x7fbe340a49a0 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1062368850 0 0) 0x7fbe340a1670 con 0x7fbe34137940 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient(hunting): my global_id is 4424 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe340a2c70 con 0x7fbe34137940 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1637321195 0 0) 0x7fbe20003520 con 0x7fbe340a41b0 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 >> v1:172.21.15.81:6791/0 conn(0x7fbe34137940 legacy=0x7fbe34137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 >> v1:172.21.15.81:6789/0 conn(0x7fbe340a49a0 legacy=0x7fbe3413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 monclient: found mon.noname-b 2024-05-08T15:47:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe340a39f0 con 0x7fbe340a41b0 2024-05-08T15:47:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe2c001ab0 con 0x7fbe340a41b0 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 5 monclient: authenticate success, global_id 74369 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: dump: 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.940 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbe2c001d10 con 0x7fbe340a41b0 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 1 -- 172.21.15.7:0/1437851280 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe2c004f60 con 0x7fbe340a41b0 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 4 set_mon_vals no callback set 2024-05-08T15:47:16.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe2bfff640 10 monclient: dump: 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.942 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 10 monclient: shutdown 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 1 -- 172.21.15.7:0/1437851280 >> v1:172.21.15.7:6790/0 conn(0x7fbe340a41b0 legacy=0x7fbe34140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 1 -- 172.21.15.7:0/1437851280 shutdown_connections 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 1 -- 172.21.15.7:0/1437851280 >> 172.21.15.7:0/1437851280 conn(0x7fbe340a9fa0 msgr2=0x7fbe340aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.930+0000 7fbe3b932640 1 -- 172.21.15.7:0/1437851280 shutdown_connections 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.931+0000 7fbe3b932640 1 -- 172.21.15.7:0/1437851280 wait complete. 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.931+0000 7fbe3b932640 10 monclient: build_initial_monmap 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.931+0000 7fbe3b932640 10 monclient: monmap: 2024-05-08T15:47:16.943 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.931+0000 7fbe3b932640 1 Processor -- start 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.931+0000 7fbe3b932640 1 -- start start 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.931+0000 7fbe3b932640 10 monclient: init 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 10 monclient: _add_conns ranks=[6,2,3,5,1,0,7,4,8] 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 10 monclient(hunting): picked mon.noname-d con 0x7fbe340a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 10 monclient(hunting): picked mon.noname-b con 0x7fbe340a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 10 monclient(hunting): picked mon.noname-g con 0x7fbe34137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe34135540 con 0x7fbe340a49a0 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe341cda40 con 0x7fbe34137940 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe341357b0 con 0x7fbe340a41b0 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe3b932640 10 monclient(hunting): authenticate will time out at 845.047075s 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe396ce640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fbe340a41b0 0x7fbe341ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56980/0 (socket says 172.21.15.7:56980) 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe396ce640 1 -- 172.21.15.7:0/407199254 learned_addr learned my addr 172.21.15.7:0/407199254 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2478692343 0 0) 0x7fbe34135540 con 0x7fbe340a49a0 2024-05-08T15:47:16.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 74378 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe18003520 con 0x7fbe340a49a0 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4188970210 0 0) 0x7fbe341357b0 con 0x7fbe340a41b0 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 74427 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe34135540 con 0x7fbe340a41b0 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.932+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2713342223 0 0) 0x7fbe18003520 con 0x7fbe340a49a0 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe341357b0 con 0x7fbe340a49a0 2024-05-08T15:47:16.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe24001980 con 0x7fbe340a49a0 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.947 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902124797 0 0) 0x7fbe341cda40 con 0x7fbe34137940 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): my global_id is 74415 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe18003520 con 0x7fbe34137940 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2181104090 0 0) 0x7fbe34135540 con 0x7fbe340a41b0 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe341cda40 con 0x7fbe340a41b0 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe300019c0 con 0x7fbe340a41b0 2024-05-08T15:47:16.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.949 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3900208664 0 0) 0x7fbe341357b0 con 0x7fbe340a49a0 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 >> v1:172.21.15.7:6792/0 conn(0x7fbe340a41b0 legacy=0x7fbe341ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 >> v1:172.21.15.81:6790/0 conn(0x7fbe34137940 legacy=0x7fbe341d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 monclient: found mon.b 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe34135a90 con 0x7fbe340a49a0 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe3b932640 5 monclient: authenticate success, global_id 74378 2024-05-08T15:47:16.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe3b932640 10 monclient: _renew_subs 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe3b932640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbe34133ef0 con 0x7fbe340a49a0 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbe24001ca0 con 0x7fbe340a49a0 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe24003c00 con 0x7fbe340a49a0 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe29ffb640 10 monclient: dump: 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:16.951 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:16.952 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:16.955 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:16.955 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:16.955 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:16.955 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe3b932640 10 monclient: _renew_subs 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe2b7fe640 4 set_mon_vals no callback set 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe3b932640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.933+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fbe34136530 con 0x7fbe340a49a0 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.934+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fbe2400ef80 con 0x7fbe340a49a0 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.934+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7fbe24045a50 con 0x7fbe340a49a0 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.934+0000 7fbe3b932640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.934+0000 7fbe3b932640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.934+0000 7fbe3b932640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.934+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe340753d0 con 0x7fbe340a49a0 2024-05-08T15:47:16.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.934+0000 7fbe396ce640 10 monclient: get_auth_request con 0x7fbe1803a1e0 auth_method 0 2024-05-08T15:47:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.939+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fbe24014be0 con 0x7fbe340a49a0 2024-05-08T15:47:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.939+0000 7fbe29ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:16.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:16.939+0000 7fbe29ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:17.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.118+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe34141a90 con 0x7fbe1803a1e0 2024-05-08T15:47:17.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.120+0000 7fbe29ffb640 1 -- 172.21.15.7:0/407199254 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33864 (unknown 2965378022 0 1755979603) 0x7fbe34141a90 con 0x7fbe1803a1e0 2024-05-08T15:47:17.117 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:47:17.117 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:47:17.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 >> v1:172.21.15.81:6812/847627118 conn(0x7fbe1803a1e0 legacy=0x7fbe1803c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 10 monclient: shutdown 2024-05-08T15:47:17.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 >> v1:172.21.15.7:6790/0 conn(0x7fbe340a49a0 legacy=0x7fbe341cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 shutdown_connections 2024-05-08T15:47:17.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 >> 172.21.15.7:0/407199254 conn(0x7fbe340a9fa0 msgr2=0x7fbe34130bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 shutdown_connections 2024-05-08T15:47:17.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.122+0000 7fbe3b932640 1 -- 172.21.15.7:0/407199254 wait complete. 2024-05-08T15:47:17.129 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-05-08T15:47:17.098867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":167268,"kb_used_data":976,"kb_used_omap":9,"kb_used_meta":166198,"kb_avail":566063772,"statfs":{"total":579820584960,"available":579649302528,"internally_reserved":0,"allocated":999424,"data_stored":1324460,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":170187426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.937415"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172443+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:47:12.172443+0000","last_peered":"2024-05-08T15:47:12.172443+0000","last_clean":"2024-05-08T15:47:12.172443+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:47:12.172443+0000","last_undegraded":"2024-05-08T15:47:12.172443+0000","last_fullsized":"2024-05-08T15:47:12.172443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:14.941756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172367+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:47:12.172367+0000","last_peered":"2024-05-08T15:47:12.172367+0000","last_clean":"2024-05-08T15:47:12.172367+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:47:12.172367+0000","last_undegraded":"2024-05-08T15:47:12.172367+0000","last_fullsized":"2024-05-08T15:47:12.172367+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:28.945664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.173944+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:30.537193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171573+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:47:12.171573+0000","last_peered":"2024-05-08T15:47:12.171573+0000","last_clean":"2024-05-08T15:47:12.171573+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:47:12.171573+0000","last_undegraded":"2024-05-08T15:47:12.171573+0000","last_fullsized":"2024-05-08T15:47:12.171573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171867+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:47:12.171867+0000","last_peered":"2024-05-08T15:47:12.171867+0000","last_clean":"2024-05-08T15:47:12.171867+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:47:12.171867+0000","last_undegraded":"2024-05-08T15:47:12.171867+0000","last_fullsized":"2024-05-08T15:47:12.171867+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171427+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:47:12.171427+0000","last_peered":"2024-05-08T15:47:12.171427+0000","last_clean":"2024-05-08T15:47:12.171427+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:47:12.171427+0000","last_undegraded":"2024-05-08T15:47:12.171427+0000","last_fullsized":"2024-05-08T15:47:12.171427+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:49.677393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.171045+0000","last_change":"2024-05-08T15:47:11.176829+0000","last_active":"2024-05-08T15:47:12.171045+0000","last_peered":"2024-05-08T15:47:12.171045+0000","last_clean":"2024-05-08T15:47:12.171045+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:47:12.171045+0000","last_undegraded":"2024-05-08T15:47:12.171045+0000","last_fullsized":"2024-05-08T15:47:12.171045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:32.476618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172416+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:47:12.172416+0000","last_peered":"2024-05-08T15:47:12.172416+0000","last_clean":"2024-05-08T15:47:12.172416+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:47:12.172416+0000","last_undegraded":"2024-05-08T15:47:12.172416+0000","last_fullsized":"2024-05-08T15:47:12.172416+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:30.243149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-05-08T15:47:12.172345+0000","last_change":"2024-05-08T15:47:04.948326+0000","last_active":"2024-05-08T15:47:12.172345+0000","last_peered":"2024-05-08T15:47:12.172345+0000","last_clean":"2024-05-08T15:47:12.172345+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:47:12.172345+0000","last_undegraded":"2024-05-08T15:47:12.172345+0000","last_fullsized":"2024-05-08T15:47:12.172345+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_clean_scrub_stamp":"2024-05-08T15:47:03.939015+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:25.024655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":47804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640260,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640261,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":147456,"data_stored":43760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640260,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":200704,"data_stored":572688,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:47:17.130 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-05-08T15:47:17.130 DEBUG:tasks.mon_thrash:Yielding 2024-05-08T15:47:17.130 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T15:47:17.139 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T15:47:17.139 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:47:17.140 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:47:17.140 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-05-08T15:47:17.140 INFO:tasks.mon_thrash.mon_thrasher:seed: 1715183237, revive delay: 90.0, thrash delay: 1.0 thrash many: True, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-05-08T15:47:17.140 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:47:17.140 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:47:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.250+0000 7f284dee6640 10 monclient: get_monmap_and_config 2024-05-08T15:47:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.250+0000 7f284dee6640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.250+0000 7f284dee6640 10 monclient: monmap: 2024-05-08T15:47:17.246 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:17.251279+0000 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:17.251279+0000 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.247 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 1 Processor -- start 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 1 -- start start 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient: init 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient: _add_conns ranks=[0,2,7,1,8,4,3,6,5] 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient(hunting): picked mon.noname-a con 0x7f28480a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient(hunting): picked mon.noname-b con 0x7f28480a5780 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient(hunting): picked mon.noname-i con 0x7f284813e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:17.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28480a1660 con 0x7f28480a53a0 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28480a28f0 con 0x7f28480a5780 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28480a2bd0 con 0x7f284813e310 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f284dee6640 10 monclient(hunting): authenticate will time out at 845.366206s 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f2846ffd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f28480a5780 0x7f284813cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:32932/0 (socket says 172.21.15.7:32932) 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f2846ffd640 1 -- 172.21.15.7:0/2270943445 learned_addr learned my addr 172.21.15.7:0/2270943445 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4208845876 0 0) 0x7f28480a28f0 con 0x7f28480a5780 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f28467fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f28467fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f28467fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.251+0000 7f28467fc640 10 monclient(hunting): my global_id is 74396 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2830003520 con 0x7f28480a5780 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3659610813 0 0) 0x7f28480a1660 con 0x7f28480a53a0 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): my global_id is 4446 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28480a28f0 con 0x7f28480a53a0 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465330711 0 0) 0x7f28480a2bd0 con 0x7f284813e310 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient(hunting): my global_id is 4417 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28480a1660 con 0x7f284813e310 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2759753771 0 0) 0x7f28480a28f0 con 0x7f28480a53a0 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 >> v1:172.21.15.81:6792/0 conn(0x7f284813e310 legacy=0x7f2848140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 >> v1:172.21.15.7:6790/0 conn(0x7f28480a5780 legacy=0x7f284813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 monclient: found mon.noname-a 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28480a6730 con 0x7f28480a53a0 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 5 monclient: authenticate success, global_id 4446 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2838001b10 con 0x7f28480a53a0 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: dump: 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.258 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:17.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2838001df0 con 0x7f28480a53a0 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 1 -- 172.21.15.7:0/2270943445 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2838003cd0 con 0x7f28480a53a0 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 4 set_mon_vals no callback set 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f28467fc640 10 monclient: dump: 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.260 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 10 monclient: shutdown 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 1 -- 172.21.15.7:0/2270943445 >> v1:172.21.15.7:6789/0 conn(0x7f28480a53a0 legacy=0x7f28480a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 1 -- 172.21.15.7:0/2270943445 shutdown_connections 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.252+0000 7f284dee6640 1 -- 172.21.15.7:0/2270943445 >> 172.21.15.7:0/2270943445 conn(0x7f28480a9fa0 msgr2=0x7f28480aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.253+0000 7f284dee6640 1 -- 172.21.15.7:0/2270943445 shutdown_connections 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.253+0000 7f284dee6640 1 -- 172.21.15.7:0/2270943445 wait complete. 2024-05-08T15:47:17.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.253+0000 7f284dee6640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.253+0000 7f284dee6640 10 monclient: monmap: 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.262 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.253+0000 7f284dee6640 1 Processor -- start 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 1 -- start start 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient: init 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient: _add_conns ranks=[5,6,3,7,1,8,4,2,0] 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient(hunting): picked mon.noname-h con 0x7f28480a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient(hunting): picked mon.noname-d con 0x7f28480a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient(hunting): picked mon.noname-g con 0x7f284813e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28481cc880 con 0x7f284813e310 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2848134e50 con 0x7f28480a53a0 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2848135110 con 0x7f28480a5780 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f284dee6640 10 monclient(hunting): authenticate will time out at 845.369171s 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f2846ffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f28480a5780 0x7f28481d1510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56992/0 (socket says 172.21.15.7:56992) 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f2846ffd640 1 -- 172.21.15.7:0/2543161979 learned_addr learned my addr 172.21.15.7:0/2543161979 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880444858 0 0) 0x7f2848135110 con 0x7f28480a5780 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f282ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f282ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f282ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f282ffff640 10 monclient(hunting): my global_id is 74454 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.254+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2834003550 con 0x7f28480a5780 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2960203521 0 0) 0x7f2834003550 con 0x7f28480a5780 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2848135110 con 0x7f28480a5780 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f283c001590 con 0x7f28480a5780 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:17.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): dump: 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.265 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68804931 0 0) 0x7f28481cc880 con 0x7f284813e310 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): my global_id is 74433 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2834003550 con 0x7f284813e310 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2052339092 0 0) 0x7f2848134e50 con 0x7f28480a53a0 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient(hunting): my global_id is 4460 2024-05-08T15:47:17.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28481cc880 con 0x7f28480a53a0 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1969739823 0 0) 0x7f2848135110 con 0x7f28480a5780 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 >> v1:172.21.15.81:6791/0 conn(0x7f28480a53a0 legacy=0x7f2848137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 >> v1:172.21.15.81:6790/0 conn(0x7f284813e310 legacy=0x7f28481d4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 monclient: found mon.d 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28481352e0 con 0x7f28480a5780 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f284dee6640 5 monclient: authenticate success, global_id 74454 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f284dee6640 10 monclient: _renew_subs 2024-05-08T15:47:17.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f284dee6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f28481384e0 con 0x7f28480a5780 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f284dee6640 10 monclient: _renew_subs 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f284dee6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f28481cc1d0 con 0x7f28480a5780 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f283c004a90 con 0x7f28480a5780 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f283c004f80 con 0x7f28480a5780 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f282ffff640 10 monclient: dump: 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.268 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.255+0000 7f2845ffb640 4 set_mon_vals no callback set 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.256+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f283c010300 con 0x7f28480a5780 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.256+0000 7f284dee6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.256+0000 7f284dee6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.256+0000 7f284dee6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.256+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2848141da0 con 0x7f28480a5780 2024-05-08T15:47:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.256+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f283c046cd0 con 0x7f28480a5780 2024-05-08T15:47:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.256+0000 7f28477fe640 10 monclient: get_auth_request con 0x7f2834039d50 auth_method 0 2024-05-08T15:47:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.261+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f283c015f40 con 0x7f28480a5780 2024-05-08T15:47:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.261+0000 7f282ffff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.261+0000 7f282ffff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:17.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.519+0000 7f284dee6640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:47:17.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.519+0000 7f284dee6640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:17.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.519+0000 7f284dee6640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.519+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2848075820 con 0x7f28480a5780 2024-05-08T15:47:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.519+0000 7f282ffff640 1 -- 172.21.15.7:0/2543161979 <== mon.6 v1:172.21.15.7:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2703 (unknown 254746657 0 1744893853) 0x7f283c015840 con 0x7f28480a5780 2024-05-08T15:47:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.519+0000 7f282ffff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:17.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.519+0000 7f282ffff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:17.516 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":10,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:47:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.521+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 >> v1:172.21.15.81:6812/847627118 conn(0x7f2834039d50 legacy=0x7f283403c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.521+0000 7f284dee6640 10 monclient: shutdown 2024-05-08T15:47:17.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.521+0000 7f284dee6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.521+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 >> v1:172.21.15.7:6792/0 conn(0x7f28480a5780 legacy=0x7f28481d1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.522+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 shutdown_connections 2024-05-08T15:47:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.522+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 >> 172.21.15.7:0/2543161979 conn(0x7f28480a9fa0 msgr2=0x7f2848131770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.522+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 shutdown_connections 2024-05-08T15:47:17.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.522+0000 7f284dee6640 1 -- 172.21.15.7:0/2543161979 wait complete. 2024-05-08T15:47:17.529 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:47:17.529 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T15:47:17.529 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:47:17.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.636+0000 7ffb9491c640 10 monclient: get_monmap_and_config 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.636+0000 7ffb9491c640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.636+0000 7ffb9491c640 10 monclient: monmap: 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:17.637741+0000 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:17.637741+0000 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.633 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.634 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.637+0000 7ffb9491c640 1 Processor -- start 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.637+0000 7ffb9491c640 1 -- start start 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.637+0000 7ffb9491c640 10 monclient: init 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.637+0000 7ffb9491c640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.637+0000 7ffb9491c640 10 monclient: _add_conns ranks=[3,5,8,2,1,0,7,6,4] 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 10 monclient(hunting): picked mon.noname-g con 0x7ffb90137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 10 monclient(hunting): picked mon.noname-h con 0x7ffb900a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 10 monclient(hunting): picked mon.noname-e con 0x7ffb900a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb900a2c70 con 0x7ffb90137940 2024-05-08T15:47:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb900a2f50 con 0x7ffb900a49a0 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb900a1670 con 0x7ffb900a41b0 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 10 monclient(hunting): authenticate will time out at 845.752682s 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8ed9d640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7ffb900a41b0 0x7ffb90140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54928/0 (socket says 172.21.15.7:54928) 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8ed9d640 1 -- 172.21.15.7:0/2472667884 learned_addr learned my addr 172.21.15.7:0/2472667884 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3678343801 0 0) 0x7ffb900a1670 con 0x7ffb900a41b0 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): my global_id is 4391 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb7c003520 con 0x7ffb900a41b0 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3846624603 0 0) 0x7ffb900a2c70 con 0x7ffb90137940 2024-05-08T15:47:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): my global_id is 74442 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb900a1670 con 0x7ffb90137940 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750592325 0 0) 0x7ffb900a2f50 con 0x7ffb900a49a0 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient(hunting): my global_id is 4469 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb900a2c70 con 0x7ffb900a49a0 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2085344748 0 0) 0x7ffb7c003520 con 0x7ffb900a41b0 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 >> v1:172.21.15.81:6791/0 conn(0x7ffb900a49a0 legacy=0x7ffb9013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 >> v1:172.21.15.81:6790/0 conn(0x7ffb90137940 legacy=0x7ffb90137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 monclient: found mon.noname-e 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb900a39f0 con 0x7ffb900a41b0 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffb88001b30 con 0x7ffb900a41b0 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: dump: 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 5 monclient: authenticate success, global_id 4391 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffb88001e50 con 0x7ffb900a41b0 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 1 -- 172.21.15.7:0/2472667884 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffb88004f60 con 0x7ffb900a41b0 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb9491c640 4 set_mon_vals no callback set 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.638+0000 7ffb8dd9b640 10 monclient: dump: 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.647 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.639+0000 7ffb9491c640 10 monclient: shutdown 2024-05-08T15:47:17.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.639+0000 7ffb9491c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.639+0000 7ffb9491c640 1 -- 172.21.15.7:0/2472667884 >> v1:172.21.15.7:6793/0 conn(0x7ffb900a41b0 legacy=0x7ffb90140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.639+0000 7ffb9491c640 1 -- 172.21.15.7:0/2472667884 shutdown_connections 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.639+0000 7ffb9491c640 1 -- 172.21.15.7:0/2472667884 >> 172.21.15.7:0/2472667884 conn(0x7ffb900a9fa0 msgr2=0x7ffb900aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.639+0000 7ffb9491c640 1 -- 172.21.15.7:0/2472667884 shutdown_connections 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.639+0000 7ffb9491c640 1 -- 172.21.15.7:0/2472667884 wait complete. 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 10 monclient: monmap: 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 1 Processor -- start 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 1 -- start start 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 10 monclient: init 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 10 monclient: _add_conns ranks=[7,3,4,5,6,8,1,0,2] 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 10 monclient(hunting): picked mon.noname-i con 0x7ffb900a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.640+0000 7ffb9491c640 10 monclient(hunting): picked mon.noname-g con 0x7ffb900a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 10 monclient(hunting): picked mon.noname-c con 0x7ffb90137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb90132e10 con 0x7ffb900a49a0 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb900a1f30 con 0x7ffb90137940 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb900a0f60 con 0x7ffb900a41b0 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 10 monclient(hunting): authenticate will time out at 845.755683s 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb8ed9d640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7ffb90137940 0x7ffb901d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:41106/0 (socket says 172.21.15.7:41106) 2024-05-08T15:47:17.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb8ed9d640 1 -- 172.21.15.7:0/1797553800 learned_addr learned my addr 172.21.15.7:0/1797553800 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180593842 0 0) 0x7ffb900a1f30 con 0x7ffb90137940 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): my global_id is 4396 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb70003520 con 0x7ffb90137940 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4141481375 0 0) 0x7ffb90132e10 con 0x7ffb900a49a0 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): my global_id is 74451 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb900a1f30 con 0x7ffb900a49a0 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3913321939 0 0) 0x7ffb70003520 con 0x7ffb90137940 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb90132e10 con 0x7ffb90137940 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffb88004ea0 con 0x7ffb90137940 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.653 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:17.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4139418586 0 0) 0x7ffb900a0f60 con 0x7ffb900a41b0 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient(hunting): my global_id is 4426 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb70003520 con 0x7ffb900a41b0 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1667702530 0 0) 0x7ffb90132e10 con 0x7ffb90137940 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.81:6792/0 conn(0x7ffb900a41b0 legacy=0x7ffb90131ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.81:6790/0 conn(0x7ffb900a49a0 legacy=0x7ffb90132700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 monclient: found mon.c 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb90137710 con 0x7ffb90137940 2024-05-08T15:47:17.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 5 monclient: authenticate success, global_id 4396 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 10 monclient: _renew_subs 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ffb9009f0c0 con 0x7ffb90137940 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 10 monclient: _renew_subs 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7ffb901d6390 con 0x7ffb90137940 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ffb88001b10 con 0x7ffb90137940 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffb88005520 con 0x7ffb90137940 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb8d59a640 4 set_mon_vals no callback set 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.641+0000 7ffb86ffd640 10 monclient: dump: 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.657 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.642+0000 7ffb9491c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.642+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ffb740065c0 con 0x7ffb74002860 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.642+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7ffb880108a0 con 0x7ffb90137940 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.642+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7ffb88047070 con 0x7ffb90137940 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942351604 0 0) 0x7ffb740065c0 con 0x7ffb74002860 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient(hunting): my global_id is 4455 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb90132e10 con 0x7ffb74002860 2024-05-08T15:47:17.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb87fff640 10 monclient: get_auth_request con 0x7ffb70039cb0 auth_method 0 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 74241604 0 0) 0x7ffb90132e10 con 0x7ffb74002860 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb740065c0 con 0x7ffb74002860 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffb80001840 con 0x7ffb74002860 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient: dump: 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.659 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 326137313 0 0) 0x7ffb740065c0 con 0x7ffb74002860 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb7400a150 con 0x7ffb74002860 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ffb80001c70 con 0x7ffb74002860 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.7:6789/0 conn(0x7ffb74002860 legacy=0x7ffb74004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.643+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.7:6789/0 conn(0x7ffb74002860 legacy=0x7ffb74004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:17.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb9491c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:17.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ffb74009e40 con 0x7ffb74009100 2024-05-08T15:47:17.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 415944450 0 0) 0x7ffb74009e40 con 0x7ffb74009100 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 10 monclient(hunting): my global_id is 4464 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb740065c0 con 0x7ffb74009100 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2591664320 0 0) 0x7ffb740065c0 con 0x7ffb74009100 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ffb74009e40 con 0x7ffb74009100 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ffb80003ba0 con 0x7ffb74009100 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 10 monclient: dump: 2024-05-08T15:47:17.663 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.664 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.665 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3180057096 0 0) 0x7ffb74009e40 con 0x7ffb74009100 2024-05-08T15:47:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.650+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffb7400c3d0 con 0x7ffb74009100 2024-05-08T15:47:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7695 (unknown 0 0 3056645691) 0x7ffb80003f20 con 0x7ffb74009100 2024-05-08T15:47:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb86ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb86ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:17.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.7:6789/0 conn(0x7ffb74009100 legacy=0x7ffb74009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb86ffd640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.7:6789/0 conn(0x7ffb74009100 legacy=0x7ffb74009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 29966, 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:17.666 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 23, 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:17.667 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:17.668 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:17.669 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:17.670 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:17.671 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:17.675 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.676 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:17.677 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.678 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.679 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:17.680 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.681 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.684 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.685 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.686 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.688 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:17.688 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.81:6812/847627118 conn(0x7ffb70039cb0 legacy=0x7ffb7003c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 10 monclient: shutdown 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 >> v1:172.21.15.7:6791/0 conn(0x7ffb90137940 legacy=0x7ffb901d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb8ed9d640 1 -- 172.21.15.7:0/1797553800 reap_dead start 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb8ed9d640 1 -- 172.21.15.7:0/1797553800 reap_dead start 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 shutdown_connections 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 >> 172.21.15.7:0/1797553800 conn(0x7ffb900a9fa0 msgr2=0x7ffb900a8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 shutdown_connections 2024-05-08T15:47:17.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.651+0000 7ffb9491c640 1 -- 172.21.15.7:0/1797553800 wait complete. 2024-05-08T15:47:17.690 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:17.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:47:17.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.798+0000 7f5b0c2a1640 10 monclient: get_monmap_and_config 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.798+0000 7f5b0c2a1640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.798+0000 7f5b0c2a1640 10 monclient: monmap: 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:17.799248+0000 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:17.799248+0000 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.798+0000 7f5b0c2a1640 1 Processor -- start 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.798+0000 7f5b0c2a1640 1 -- start start 2024-05-08T15:47:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.798+0000 7f5b0c2a1640 10 monclient: init 2024-05-08T15:47:17.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 10 monclient: _add_conns ranks=[6,2,0,3,5,8,4,7,1] 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 10 monclient(hunting): picked mon.noname-d con 0x7f5b040a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 10 monclient(hunting): picked mon.noname-b con 0x7f5b040a5780 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 10 monclient(hunting): picked mon.noname-a con 0x7f5b0413e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b040a1660 con 0x7f5b0413e310 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b040a28f0 con 0x7f5b040a5780 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b040a2bd0 con 0x7f5b040a53a0 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0c2a1640 10 monclient(hunting): authenticate will time out at 845.914196s 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0983c640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f5b040a5780 0x7f5b0413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:32948/0 (socket says 172.21.15.7:32948) 2024-05-08T15:47:17.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0983c640 1 -- 172.21.15.7:0/3153779283 learned_addr learned my addr 172.21.15.7:0/3153779283 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0a83e640 1 --1- 172.21.15.7:0/3153779283 >> v1:172.21.15.7:6789/0 conn(0x7f5b0413e310 0x7f5b04140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58190/0 (socket says 172.21.15.7:58190) 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2143629741 0 0) 0x7f5b040a28f0 con 0x7f5b040a5780 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0903b640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0903b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0903b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0903b640 10 monclient(hunting): my global_id is 74405 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.799+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5af0003520 con 0x7f5b040a5780 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197754881 0 0) 0x7f5b040a2bd0 con 0x7f5b040a53a0 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): my global_id is 74463 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b040a28f0 con 0x7f5b040a53a0 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907180531 0 0) 0x7f5b040a1660 con 0x7f5b0413e310 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient(hunting): my global_id is 4473 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b040a2bd0 con 0x7f5b0413e310 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 623121667 0 0) 0x7f5af0003520 con 0x7f5b040a5780 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 >> v1:172.21.15.7:6792/0 conn(0x7f5b040a53a0 legacy=0x7f5b040a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 >> v1:172.21.15.7:6789/0 conn(0x7f5b0413e310 legacy=0x7f5b04140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 monclient: found mon.noname-b 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b040a6730 con 0x7f5b040a5780 2024-05-08T15:47:17.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5af40017b0 con 0x7f5b040a5780 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: dump: 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.809 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0c2a1640 5 monclient: authenticate success, global_id 74405 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0c2a1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5af4001ad0 con 0x7f5b040a5780 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 1 -- 172.21.15.7:0/3153779283 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5af4004be0 con 0x7f5b040a5780 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0c2a1640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0c2a1640 4 set_mon_vals no callback set 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0903b640 10 monclient: dump: 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.811 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0c2a1640 10 monclient: shutdown 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0c2a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.800+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/3153779283 >> v1:172.21.15.7:6790/0 conn(0x7f5b040a5780 legacy=0x7f5b0413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/3153779283 shutdown_connections 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/3153779283 >> 172.21.15.7:0/3153779283 conn(0x7f5b040a9fa0 msgr2=0x7f5b040aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/3153779283 shutdown_connections 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/3153779283 wait complete. 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 10 monclient: monmap: 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.813 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 1 Processor -- start 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.801+0000 7f5b0c2a1640 1 -- start start 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient: init 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient: _add_conns ranks=[1,3,7,0,8,6,2,5,4] 2024-05-08T15:47:17.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient(hunting): picked mon.noname-f con 0x7f5b040a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient(hunting): picked mon.noname-g con 0x7f5b040a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient(hunting): picked mon.noname-i con 0x7f5b0413e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b040a43c0 con 0x7f5b040a53a0 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b04137880 con 0x7f5b040a5780 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b041363d0 con 0x7f5b0413e310 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0c2a1640 10 monclient(hunting): authenticate will time out at 845.917052s 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0983c640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f5b040a5780 0x7f5b041d1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:36990/0 (socket says 172.21.15.7:36990) 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5b0983c640 1 -- 172.21.15.7:0/1187344780 learned_addr learned my addr 172.21.15.7:0/1187344780 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399566514 0 0) 0x7f5b04137880 con 0x7f5b040a5780 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): my global_id is 74460 2024-05-08T15:47:17.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ae8003520 con 0x7f5b040a5780 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265487526 0 0) 0x7f5b041363d0 con 0x7f5b0413e310 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.802+0000 7f5afaffd640 10 monclient(hunting): my global_id is 4435 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b04137880 con 0x7f5b0413e310 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3402865876 0 0) 0x7f5b040a28f0 con 0x7f5b040a53a0 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): my global_id is 4420 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b041363d0 con 0x7f5b040a53a0 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1184380894 0 0) 0x7f5ae8003520 con 0x7f5b040a5780 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b040a28f0 con 0x7f5b040a5780 2024-05-08T15:47:17.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5af4004b20 con 0x7f5b040a5780 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.817 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 657603783 0 0) 0x7f5b04137880 con 0x7f5b0413e310 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5ae8003520 con 0x7f5b0413e310 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1521716061 0 0) 0x7f5b041363d0 con 0x7f5b040a53a0 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5b04137880 con 0x7f5b040a53a0 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5afc001c30 con 0x7f5b0413e310 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.819 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5b00001840 con 0x7f5b040a53a0 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient(hunting): dump: 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.820 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.821 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.822 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 980045275 0 0) 0x7f5b040a28f0 con 0x7f5b040a5780 2024-05-08T15:47:17.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.81:6792/0 conn(0x7f5b0413e310 legacy=0x7f5b041d4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.81:6789/0 conn(0x7f5b040a53a0 legacy=0x7f5b04131ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 monclient: found mon.g 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b04135870 con 0x7f5b040a5780 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0c2a1640 5 monclient: authenticate success, global_id 74460 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0c2a1640 10 monclient: _renew_subs 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0c2a1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5b04136610 con 0x7f5b040a5780 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0c2a1640 10 monclient: _renew_subs 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0c2a1640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5b041323b0 con 0x7f5b040a5780 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5af4001ab0 con 0x7f5b040a5780 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5af4001ce0 con 0x7f5b040a5780 2024-05-08T15:47:17.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5b0883a640 4 set_mon_vals no callback set 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.803+0000 7f5afaffd640 10 monclient: dump: 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.825 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5b0c2a1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5aec0065c0 con 0x7f5aec002860 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f5af40106c0 con 0x7f5b040a5780 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f5af4046fb0 con 0x7f5b040a5780 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032643973 0 0) 0x7f5aec0065c0 con 0x7f5aec002860 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 10 monclient(hunting): my global_id is 74414 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b040a28f0 con 0x7f5aec002860 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.804+0000 7f5b0a83e640 10 monclient: get_auth_request con 0x7f5ae803a400 auth_method 0 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3402344979 0 0) 0x7f5b040a28f0 con 0x7f5aec002860 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5aec0065c0 con 0x7f5aec002860 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5b00001c10 con 0x7f5aec002860 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 10 monclient: dump: 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.827 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3493904231 0 0) 0x7f5aec0065c0 con 0x7f5aec002860 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5aec00a150 con 0x7f5aec002860 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5b000059f0 con 0x7f5aec002860 2024-05-08T15:47:17.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.7:6790/0 conn(0x7f5aec002860 legacy=0x7f5aec004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.805+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.7:6790/0 conn(0x7f5aec002860 legacy=0x7f5aec004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5b0c2a1640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5aec009e40 con 0x7f5aec009100 2024-05-08T15:47:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805878723 0 0) 0x7f5aec009e40 con 0x7f5aec009100 2024-05-08T15:47:17.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 10 monclient(hunting): my global_id is 74423 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5aec0065c0 con 0x7f5aec009100 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1003262924 0 0) 0x7f5aec0065c0 con 0x7f5aec009100 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5aec009e40 con 0x7f5aec009100 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5b00004110 con 0x7f5aec009100 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 10 monclient: dump: 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.831 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3935675992 0 0) 0x7f5aec009e40 con 0x7f5aec009100 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.812+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5aec00c3d0 con 0x7f5aec009100 2024-05-08T15:47:17.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 3258515943) 0x7f5b00005ba0 con 0x7f5aec009100 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5afaffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5afaffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.7:6790/0 conn(0x7f5aec009100 legacy=0x7f5aec009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5afaffd640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.7:6790/0 conn(0x7f5aec009100 legacy=0x7f5aec009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 30110, 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:17.833 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 23, 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:17.834 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:17.835 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:17.836 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:17.837 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.838 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:17.842 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.843 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.844 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:17.845 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.848 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:17.849 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:17.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:17.851 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:17.852 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.81:6812/847627118 conn(0x7f5ae803a400 legacy=0x7f5ae803c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 10 monclient: shutdown 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 >> v1:172.21.15.81:6790/0 conn(0x7f5b040a5780 legacy=0x7f5b041d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0a83e640 1 -- 172.21.15.7:0/1187344780 reap_dead start 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0a83e640 1 -- 172.21.15.7:0/1187344780 reap_dead start 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 shutdown_connections 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 >> 172.21.15.7:0/1187344780 conn(0x7f5b040a9fa0 msgr2=0x7f5b040a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 shutdown_connections 2024-05-08T15:47:17.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.813+0000 7f5b0c2a1640 1 -- 172.21.15.7:0/1187344780 wait complete. 2024-05-08T15:47:17.854 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:17.854 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:47:17.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:47:17.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.964+0000 7f807f8ac640 10 monclient: get_monmap_and_config 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.964+0000 7f807f8ac640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.964+0000 7f807f8ac640 10 monclient: monmap: 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:17.965458+0000 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:17.965458+0000 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.961 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.962 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.962 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.962 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.962 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 1 Processor -- start 2024-05-08T15:47:17.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 1 -- start start 2024-05-08T15:47:17.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient: init 2024-05-08T15:47:17.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient: _add_conns ranks=[6,8,3,2,7,4,1,0,5] 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient(hunting): picked mon.noname-d con 0x7f8078137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient(hunting): picked mon.noname-e con 0x7f80780a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient(hunting): picked mon.noname-g con 0x7f80780a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80780a2c70 con 0x7f80780a41b0 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80780a2f50 con 0x7f8078137940 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80780a1670 con 0x7f80780a49a0 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807f8ac640 10 monclient(hunting): authenticate will time out at 846.080424s 2024-05-08T15:47:17.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807d648640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f8078137940 0x7f8078137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:57010/0 (socket says 172.21.15.7:57010) 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807ce47640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f80780a49a0 0x7f807813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54930/0 (socket says 172.21.15.7:54930) 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.965+0000 7f807d648640 1 -- 172.21.15.7:0/1506513869 learned_addr learned my addr 172.21.15.7:0/1506513869 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3196019481 0 0) 0x7f80780a1670 con 0x7f80780a49a0 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): my global_id is 4400 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8064003520 con 0x7f80780a49a0 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3289575210 0 0) 0x7f80780a2f50 con 0x7f8078137940 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): my global_id is 74472 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80780a1670 con 0x7f8078137940 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147332634 0 0) 0x7f80780a2c70 con 0x7f80780a41b0 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient(hunting): my global_id is 74469 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80780a2f50 con 0x7f80780a41b0 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1555858118 0 0) 0x7f8064003520 con 0x7f80780a49a0 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 >> v1:172.21.15.7:6792/0 conn(0x7f8078137940 legacy=0x7f8078137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 >> v1:172.21.15.81:6790/0 conn(0x7f80780a41b0 legacy=0x7f8078140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 monclient: found mon.noname-e 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80780a39f0 con 0x7f80780a49a0 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f80680017b0 con 0x7f80780a49a0 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f807f8ac640 5 monclient: authenticate success, global_id 4400 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: dump: 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.977 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f807f8ac640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:17.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8068001ad0 con 0x7f80780a49a0 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 1 -- 172.21.15.7:0/1506513869 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8068004b20 con 0x7f80780a49a0 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f807f8ac640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f807f8ac640 4 set_mon_vals no callback set 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.966+0000 7f806ffff640 10 monclient: dump: 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.979 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 10 monclient: shutdown 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 1 -- 172.21.15.7:0/1506513869 >> v1:172.21.15.7:6793/0 conn(0x7f80780a49a0 legacy=0x7f807813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 1 -- 172.21.15.7:0/1506513869 shutdown_connections 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 1 -- 172.21.15.7:0/1506513869 >> 172.21.15.7:0/1506513869 conn(0x7f80780a9fa0 msgr2=0x7f80780aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 1 -- 172.21.15.7:0/1506513869 shutdown_connections 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 1 -- 172.21.15.7:0/1506513869 wait complete. 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 10 monclient: build_initial_monmap 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.967+0000 7f807f8ac640 10 monclient: monmap: 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:17.981 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 1 Processor -- start 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 1 -- start start 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient: init 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient: _add_conns ranks=[6,1,0,4,8,5,2,7,3] 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient(hunting): picked mon.noname-d con 0x7f80780a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:17.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient(hunting): picked mon.noname-f con 0x7f80780a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient(hunting): picked mon.noname-a con 0x7f8078137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80781360a0 con 0x7f8078137940 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8078133070 con 0x7f80780a49a0 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80780a0f60 con 0x7f80780a41b0 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807f8ac640 10 monclient(hunting): authenticate will time out at 846.083280s 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807d648640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f80780a41b0 0x7f8078132250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:57012/0 (socket says 172.21.15.7:57012) 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f807d648640 1 -- 172.21.15.7:0/3862370826 learned_addr learned my addr 172.21.15.7:0/3862370826 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030870724 0 0) 0x7f80780a0f60 con 0x7f80780a41b0 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): my global_id is 74481 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f805c003520 con 0x7f80780a41b0 2024-05-08T15:47:17.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2527668626 0 0) 0x7f80781360a0 con 0x7f8078137940 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 10 monclient(hunting): my global_id is 4482 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.968+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80780a0f60 con 0x7f8078137940 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214196462 0 0) 0x7f8078133070 con 0x7f80780a49a0 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): my global_id is 4429 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80781360a0 con 0x7f80780a49a0 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2684600404 0 0) 0x7f805c003520 con 0x7f80780a41b0 2024-05-08T15:47:17.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8078133070 con 0x7f80780a41b0 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8074001950 con 0x7f80780a41b0 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): dump: 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.985 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 613644009 0 0) 0x7f80780a0f60 con 0x7f8078137940 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f805c003520 con 0x7f8078137940 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8070001dd0 con 0x7f8078137940 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient(hunting): dump: 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.986 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2352496579 0 0) 0x7f8078133070 con 0x7f80780a41b0 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.81:6789/0 conn(0x7f80780a49a0 legacy=0x7f8078132960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.7:6789/0 conn(0x7f8078137940 legacy=0x7f80781d5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 monclient: found mon.d 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8078135dc0 con 0x7f80780a41b0 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f807f8ac640 5 monclient: authenticate success, global_id 74481 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f807f8ac640 10 monclient: _renew_subs 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f807f8ac640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f80781354f0 con 0x7f80780a41b0 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f80740047e0 con 0x7f80780a41b0 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8074004cd0 con 0x7f80780a41b0 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806dffb640 10 monclient: dump: 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.989 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f807f8ac640 10 monclient: _renew_subs 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f806f7fe640 4 set_mon_vals no callback set 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f807f8ac640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.969+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f8078136830 con 0x7f80780a41b0 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f8074010050 con 0x7f80780a41b0 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f807f8ac640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f80780753d0 con 0x7f807809ba50 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f8074046ad0 con 0x7f80780a41b0 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f807de49640 10 monclient: get_auth_request con 0x7f805c03a1a0 auth_method 0 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 356228488 0 0) 0x7f80780753d0 con 0x7f807809ba50 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 10 monclient(hunting): my global_id is 4405 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8078133070 con 0x7f807809ba50 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2368073236 0 0) 0x7f8078133070 con 0x7f807809ba50 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f80780753d0 con 0x7f807809ba50 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8068001430 con 0x7f807809ba50 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.970+0000 7f806dffb640 10 monclient: dump: 2024-05-08T15:47:17.991 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.992 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.971+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2091388953 0 0) 0x7f80780753d0 con 0x7f807809ba50 2024-05-08T15:47:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.971+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80780a1f30 con 0x7f807809ba50 2024-05-08T15:47:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.971+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f8068001720 con 0x7f807809ba50 2024-05-08T15:47:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.971+0000 7f806dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.971+0000 7f806dffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.971+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.7:6791/0 conn(0x7f807809ba50 legacy=0x7f807809be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.971+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.7:6791/0 conn(0x7f807809ba50 legacy=0x7f807809be30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:17.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.977+0000 7f807f8ac640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.977+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8078136ac0 con 0x7f807809c540 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1205552678 0 0) 0x7f8078136ac0 con 0x7f807809c540 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient(hunting): my global_id is 4414 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80780753d0 con 0x7f807809c540 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3717580235 0 0) 0x7f80780753d0 con 0x7f807809c540 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8078136ac0 con 0x7f807809c540 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8068005530 con 0x7f807809c540 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:17.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient: dump: 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:17.996 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2179437436 0 0) 0x7f8078136ac0 con 0x7f807809c540 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8078135990 con 0x7f807809c540 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7863 (unknown 0 0 3430725435) 0x7f8068007830 con 0x7f807809c540 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.7:6791/0 conn(0x7f807809c540 legacy=0x7f80781da7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.978+0000 7f806dffb640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.7:6791/0 conn(0x7f807809c540 legacy=0x7f80781da7d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:17.997 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 30286, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:17.998 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 24, 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:17.999 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.000 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:18.001 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.002 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.003 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:18.004 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.005 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.006 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.007 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:18.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.011 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.012 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.013 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:18.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.015 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:18.016 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.81:6812/847627118 conn(0x7f805c03a1a0 legacy=0x7f805c03c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 10 monclient: shutdown 2024-05-08T15:47:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 >> v1:172.21.15.7:6792/0 conn(0x7f80780a41b0 legacy=0x7f8078132250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807de49640 1 -- 172.21.15.7:0/3862370826 reap_dead start 2024-05-08T15:47:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807de49640 1 -- 172.21.15.7:0/3862370826 reap_dead start 2024-05-08T15:47:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 shutdown_connections 2024-05-08T15:47:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 >> 172.21.15.7:0/3862370826 conn(0x7f80780a9fa0 msgr2=0x7f8078130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 shutdown_connections 2024-05-08T15:47:18.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:17.979+0000 7f807f8ac640 1 -- 172.21.15.7:0/3862370826 wait complete. 2024-05-08T15:47:18.018 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:18.018 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:47:18.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.126+0000 7f7cd5b60640 10 monclient: get_monmap_and_config 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.126+0000 7f7cd5b60640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.126+0000 7f7cd5b60640 10 monclient: monmap: 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:18.127626+0000 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:18.127626+0000 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.123 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.124 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.124 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 1 Processor -- start 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 1 -- start start 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient: init 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient: _add_conns ranks=[2,3,1,8,6,4,5,7,0] 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient(hunting): picked mon.noname-b con 0x7f7cd0137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient(hunting): picked mon.noname-g con 0x7f7cd00a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient(hunting): picked mon.noname-f con 0x7f7cd00a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd00a2c70 con 0x7f7cd00a41b0 2024-05-08T15:47:18.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd00a2f50 con 0x7f7cd0137940 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd00a1670 con 0x7f7cd00a49a0 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.127+0000 7f7cd5b60640 10 monclient(hunting): authenticate will time out at 846.242585s 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7ccf7fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f7cd0137940 0x7f7cd0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:32978/0 (socket says 172.21.15.7:32978) 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7ccf7fe640 1 -- 172.21.15.7:0/1258306037 learned_addr learned my addr 172.21.15.7:0/1258306037 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 175087840 0 0) 0x7f7cd00a2f50 con 0x7f7cd0137940 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): my global_id is 74432 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cb8003520 con 0x7f7cd0137940 2024-05-08T15:47:18.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891485762 0 0) 0x7f7cd00a2c70 con 0x7f7cd00a41b0 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): my global_id is 4438 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd00a2f50 con 0x7f7cd00a41b0 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2293735611 0 0) 0x7f7cd00a1670 con 0x7f7cd00a49a0 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient(hunting): my global_id is 74478 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd00a2c70 con 0x7f7cd00a49a0 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3287940772 0 0) 0x7f7cb8003520 con 0x7f7cd0137940 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 >> v1:172.21.15.81:6790/0 conn(0x7f7cd00a49a0 legacy=0x7f7cd013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 >> v1:172.21.15.81:6789/0 conn(0x7f7cd00a41b0 legacy=0x7f7cd0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 monclient: found mon.noname-b 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd00a39f0 con 0x7f7cd0137940 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc0001730 con 0x7f7cd0137940 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cd5b60640 5 monclient: authenticate success, global_id 74432 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cd5b60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: dump: 2024-05-08T15:47:18.136 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cd5b60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7cc0001990 con 0x7f7cd0137940 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 1 -- 172.21.15.7:0/1258306037 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc0004be0 con 0x7f7cd0137940 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cce7fc640 10 monclient: dump: 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.138 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cd5b60640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.128+0000 7f7cd5b60640 4 set_mon_vals no callback set 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 10 monclient: shutdown 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 1 -- 172.21.15.7:0/1258306037 >> v1:172.21.15.7:6790/0 conn(0x7f7cd0137940 legacy=0x7f7cd0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 1 -- 172.21.15.7:0/1258306037 shutdown_connections 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 1 -- 172.21.15.7:0/1258306037 >> 172.21.15.7:0/1258306037 conn(0x7f7cd00a9fa0 msgr2=0x7f7cd00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 1 -- 172.21.15.7:0/1258306037 shutdown_connections 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 1 -- 172.21.15.7:0/1258306037 wait complete. 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.129+0000 7f7cd5b60640 10 monclient: monmap: 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.140 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 1 Processor -- start 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 1 -- start start 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient: init 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient: _add_conns ranks=[3,0,4,1,8,5,7,6,2] 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient(hunting): picked mon.noname-g con 0x7f7cd00a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient(hunting): picked mon.noname-a con 0x7f7cd00a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient(hunting): picked mon.noname-c con 0x7f7cd0137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd0135b30 con 0x7f7cd00a49a0 2024-05-08T15:47:18.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd00a1f30 con 0x7f7cd00a41b0 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cd00a0f60 con 0x7f7cd0137940 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cd5b60640 10 monclient(hunting): authenticate will time out at 846.245463s 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cceffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7cd00a49a0 0x7f7cd0132880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58206/0 (socket says 172.21.15.7:58206) 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7ccffff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f7cd0137940 0x7f7cd01d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:41154/0 (socket says 172.21.15.7:41154) 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.130+0000 7f7cceffd640 1 -- 172.21.15.7:0/2519557895 learned_addr learned my addr 172.21.15.7:0/2519557895 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207197859 0 0) 0x7f7cd00a1f30 con 0x7f7cd00a41b0 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): my global_id is 74487 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cbc003520 con 0x7f7cd00a41b0 2024-05-08T15:47:18.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 405257891 0 0) 0x7f7cd0135b30 con 0x7f7cd00a49a0 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): my global_id is 4491 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd00a1f30 con 0x7f7cd00a49a0 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273318473 0 0) 0x7f7cd00a0f60 con 0x7f7cd0137940 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): my global_id is 4423 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd0135b30 con 0x7f7cd0137940 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4016328173 0 0) 0x7f7cbc003520 con 0x7f7cd00a41b0 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7cd00a0f60 con 0x7f7cd00a41b0 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc00016f0 con 0x7f7cd00a41b0 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): dump: 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.144 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 487108362 0 0) 0x7f7cd00a1f30 con 0x7f7cd00a49a0 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7cbc003520 con 0x7f7cd00a49a0 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3087428404 0 0) 0x7f7cd0135b30 con 0x7f7cd0137940 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7cd00a1f30 con 0x7f7cd0137940 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc4001a50 con 0x7f7cd00a49a0 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): dump: 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.145 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc8001c80 con 0x7f7cd0137940 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient(hunting): dump: 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.147 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3618087448 0 0) 0x7f7cd00a0f60 con 0x7f7cd00a41b0 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.7:6791/0 conn(0x7f7cd0137940 legacy=0x7f7cd01d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.7:6789/0 conn(0x7f7cd00a49a0 legacy=0x7f7cd0132880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 monclient: found mon.g 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd0132f90 con 0x7f7cd00a41b0 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cd5b60640 5 monclient: authenticate success, global_id 74487 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cd5b60640 10 monclient: _renew_subs 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cd5b60640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd0135e10 con 0x7f7cd00a41b0 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cd5b60640 10 monclient: _renew_subs 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cd5b60640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f7cd0134830 con 0x7f7cd00a41b0 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7cc0004c20 con 0x7f7cd00a41b0 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc0005210 con 0x7f7cd00a41b0 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7ccdffb640 4 set_mon_vals no callback set 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.131+0000 7f7cb7fff640 10 monclient: dump: 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.149 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.132+0000 7f7cd5b60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.132+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7cb00065c0 con 0x7f7cb0002860 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.132+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7cc0010610 con 0x7f7cd00a41b0 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.132+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(18..18 src has 1..18) ==== 3506+0+0 (unknown 3002353116 0 0) 0x7f7cc0046cb0 con 0x7f7cd00a41b0 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2737171173 0 0) 0x7f7cb00065c0 con 0x7f7cb0002860 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient(hunting): my global_id is 74490 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd00a0f60 con 0x7f7cb0002860 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7ccffff640 10 monclient: get_auth_request con 0x7f7cbc03a3e0 auth_method 0 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 332345731 0 0) 0x7f7cd00a0f60 con 0x7f7cb0002860 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7cb00065c0 con 0x7f7cb0002860 2024-05-08T15:47:18.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc4001d40 con 0x7f7cb0002860 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient: dump: 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.152 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4226834917 0 0) 0x7f7cb00065c0 con 0x7f7cb0002860 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7cb000a150 con 0x7f7cb0002860 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7cc4003df0 con 0x7f7cb0002860 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.7:6792/0 conn(0x7f7cb0002860 legacy=0x7f7cb0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.133+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.7:6792/0 conn(0x7f7cb0002860 legacy=0x7f7cb0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cd5b60640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7cb0009e20 con 0x7f7cb00090e0 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136657440 0 0) 0x7f7cb0009e20 con 0x7f7cb00090e0 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 10 monclient(hunting): my global_id is 74499 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cb00065c0 con 0x7f7cb00090e0 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 290709930 0 0) 0x7f7cb00065c0 con 0x7f7cb00090e0 2024-05-08T15:47:18.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7cb0009e20 con 0x7f7cb00090e0 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7cc40057c0 con 0x7f7cb00090e0 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 10 monclient: dump: 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.156 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 290884845 0 0) 0x7f7cb0009e20 con 0x7f7cb00090e0 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.140+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7cb000c390 con 0x7f7cb00090e0 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 3947486302) 0x7f7cc4007aa0 con 0x7f7cb00090e0 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cb7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cb7fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.7:6792/0 conn(0x7f7cb00090e0 legacy=0x7f7cb0009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cb7fff640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.7:6792/0 conn(0x7f7cb00090e0 legacy=0x7f7cb0009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.157 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 30455, 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:18.158 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 24, 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.159 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:18.161 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.162 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.165 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.169 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.170 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.174 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:18.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.178 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:18.179 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.81:6812/847627118 conn(0x7f7cbc03a3e0 legacy=0x7f7cbc03c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 10 monclient: shutdown 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 >> v1:172.21.15.81:6790/0 conn(0x7f7cd00a41b0 legacy=0x7f7cd0132170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7ccffff640 1 -- 172.21.15.7:0/2519557895 reap_dead start 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7ccffff640 1 -- 172.21.15.7:0/2519557895 reap_dead start 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 shutdown_connections 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 >> 172.21.15.7:0/2519557895 conn(0x7f7cd00a9fa0 msgr2=0x7f7cd0130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 shutdown_connections 2024-05-08T15:47:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.141+0000 7f7cd5b60640 1 -- 172.21.15.7:0/2519557895 wait complete. 2024-05-08T15:47:18.182 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:18.182 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:47:18.182 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:47:18.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.296+0000 7f652952b640 10 monclient: get_monmap_and_config 2024-05-08T15:47:18.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.296+0000 7f652952b640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.296+0000 7f652952b640 10 monclient: monmap: 2024-05-08T15:47:18.292 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:18.297232+0000 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:18.297232+0000 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.293 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.296+0000 7f652952b640 1 Processor -- start 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.296+0000 7f652952b640 1 -- start start 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.296+0000 7f652952b640 10 monclient: init 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 10 monclient: _add_conns ranks=[5,1,3,6,2,8,7,0,4] 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 10 monclient(hunting): picked mon.noname-h con 0x7f65240a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 10 monclient(hunting): picked mon.noname-f con 0x7f65240a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 10 monclient(hunting): picked mon.noname-g con 0x7f652413e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65240a1660 con 0x7f65240a5780 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65240a28f0 con 0x7f652413e310 2024-05-08T15:47:18.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65240a2bd0 con 0x7f65240a53a0 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f652952b640 10 monclient(hunting): authenticate will time out at 846.412231s 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f65227fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f65240a5780 0x7f652413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:34158/0 (socket says 172.21.15.7:34158) 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f65237fe640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f652413e310 0x7f6524140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:37026/0 (socket says 172.21.15.7:37026) 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.297+0000 7f65227fc640 1 -- 172.21.15.7:0/2065843744 learned_addr learned my addr 172.21.15.7:0/2065843744 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1085963785 0 0) 0x7f65240a2bd0 con 0x7f65240a53a0 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): my global_id is 4478 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6510003520 con 0x7f65240a53a0 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4033013429 0 0) 0x7f65240a1660 con 0x7f65240a5780 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): my global_id is 4447 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65240a2bd0 con 0x7f65240a5780 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2180680388 0 0) 0x7f65240a28f0 con 0x7f652413e310 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient(hunting): my global_id is 74496 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65240a1660 con 0x7f652413e310 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1565332706 0 0) 0x7f6510003520 con 0x7f65240a53a0 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 >> v1:172.21.15.81:6790/0 conn(0x7f652413e310 legacy=0x7f6524140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 >> v1:172.21.15.81:6789/0 conn(0x7f65240a5780 legacy=0x7f652413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 monclient: found mon.noname-h 2024-05-08T15:47:18.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65240a6730 con 0x7f65240a53a0 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6514001a60 con 0x7f65240a53a0 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: dump: 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.306 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f652952b640 5 monclient: authenticate success, global_id 4478 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f652952b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6514001d80 con 0x7f65240a53a0 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 1 -- 172.21.15.7:0/2065843744 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6514003cd0 con 0x7f65240a53a0 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f6521ffb640 10 monclient: dump: 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.308 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f652952b640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.298+0000 7f652952b640 4 set_mon_vals no callback set 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 10 monclient: shutdown 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 1 -- 172.21.15.7:0/2065843744 >> v1:172.21.15.81:6791/0 conn(0x7f65240a53a0 legacy=0x7f65240a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 1 -- 172.21.15.7:0/2065843744 shutdown_connections 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 1 -- 172.21.15.7:0/2065843744 >> 172.21.15.7:0/2065843744 conn(0x7f65240a9fa0 msgr2=0x7f65240aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 1 -- 172.21.15.7:0/2065843744 shutdown_connections 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 1 -- 172.21.15.7:0/2065843744 wait complete. 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.299+0000 7f652952b640 10 monclient: monmap: 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.310 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 1 Processor -- start 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 1 -- start start 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient: init 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient: _add_conns ranks=[6,0,5,3,4,2,7,1,8] 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient(hunting): picked mon.noname-d con 0x7f65240a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient(hunting): picked mon.noname-a con 0x7f65240a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient(hunting): picked mon.noname-h con 0x7f652413e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6524135120 con 0x7f65240a5780 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6524134e40 con 0x7f652413e310 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f6522ffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f65240a53a0 0x7f6524137920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:57034/0 (socket says 172.21.15.7:57034) 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6524135400 con 0x7f65240a53a0 2024-05-08T15:47:18.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f6522ffd640 1 -- 172.21.15.7:0/1074765078 learned_addr learned my addr 172.21.15.7:0/1074765078 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.300+0000 7f652952b640 10 monclient(hunting): authenticate will time out at 846.415573s 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993609741 0 0) 0x7f65240a2bd0 con 0x7f65240a5780 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): my global_id is 4500 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6504003520 con 0x7f65240a5780 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330963935 0 0) 0x7f6524135400 con 0x7f65240a53a0 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): my global_id is 74508 2024-05-08T15:47:18.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65240a2bd0 con 0x7f65240a53a0 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2926259366 0 0) 0x7f6524134e40 con 0x7f652413e310 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): my global_id is 4487 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6524135400 con 0x7f652413e310 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1150271005 0 0) 0x7f65240a2bd0 con 0x7f65240a53a0 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6524134e40 con 0x7f65240a53a0 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6514001a20 con 0x7f65240a53a0 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): dump: 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.314 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 438179010 0 0) 0x7f6504003520 con 0x7f65240a5780 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f65240a2bd0 con 0x7f65240a5780 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f650c0019d0 con 0x7f65240a5780 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): dump: 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.315 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1431355579 0 0) 0x7f6524135400 con 0x7f652413e310 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6504003520 con 0x7f652413e310 2024-05-08T15:47:18.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f651c0017e0 con 0x7f652413e310 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient(hunting): dump: 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.317 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4092851836 0 0) 0x7f6524134e40 con 0x7f65240a53a0 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.81:6791/0 conn(0x7f652413e310 legacy=0x7f65241d57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.7:6789/0 conn(0x7f65240a5780 legacy=0x7f65241d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 monclient: found mon.d 2024-05-08T15:47:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652413d020 con 0x7f65240a53a0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f652952b640 5 monclient: authenticate success, global_id 74508 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f652952b640 10 monclient: _renew_subs 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f652952b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f65241cd290 con 0x7f65240a53a0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f652952b640 10 monclient: _renew_subs 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f652952b640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f65241cd580 con 0x7f65240a53a0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6514001d80 con 0x7f65240a53a0 2024-05-08T15:47:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6514004320 con 0x7f65240a53a0 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f65217fa640 4 set_mon_vals no callback set 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.301+0000 7f651b7fe640 10 monclient: dump: 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.302+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f65140044c0 con 0x7f65240a53a0 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.302+0000 7f652952b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.302+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f65080065c0 con 0x7f6508002860 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.302+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7f6514047290 con 0x7f65240a53a0 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f65237fe640 10 monclient: get_auth_request con 0x7f650403a4b0 auth_method 0 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 330125296 0 0) 0x7f65080065c0 con 0x7f6508002860 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient(hunting): my global_id is 4409 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6524134e40 con 0x7f6508002860 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2869000927 0 0) 0x7f6524134e40 con 0x7f6508002860 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f65080065c0 con 0x7f6508002860 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f650c004d10 con 0x7f6508002860 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient: dump: 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.322 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 397718860 0 0) 0x7f65080065c0 con 0x7f6508002860 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f650800a150 con 0x7f6508002860 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f650c007480 con 0x7f6508002860 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:18.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.7:6793/0 conn(0x7f6508002860 legacy=0x7f6508004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.303+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.7:6793/0 conn(0x7f6508002860 legacy=0x7f6508004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f652952b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6508009e40 con 0x7f6508009100 2024-05-08T15:47:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538594536 0 0) 0x7f6508009e40 con 0x7f6508009100 2024-05-08T15:47:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 10 monclient(hunting): my global_id is 4418 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65080065c0 con 0x7f6508009100 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1638204137 0 0) 0x7f65080065c0 con 0x7f6508009100 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6508009e40 con 0x7f6508009100 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f650c005680 con 0x7f6508009100 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 10 monclient: dump: 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.326 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1242750899 0 0) 0x7f6508009e40 con 0x7f6508009100 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.310+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f650800c3d0 con 0x7f6508009100 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1576015238) 0x7f650c007630 con 0x7f6508009100 2024-05-08T15:47:18.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f651b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f651b7fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.7:6793/0 conn(0x7f6508009100 legacy=0x7f6508009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f651b7fe640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.7:6793/0 conn(0x7f6508009100 legacy=0x7f6508009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 30640, 2024-05-08T15:47:18.328 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 24, 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:18.329 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.330 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:18.331 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:18.332 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:18.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.334 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:18.337 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.339 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:18.343 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.344 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.345 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.81:6812/847627118 conn(0x7f650403a4b0 legacy=0x7f650403c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 10 monclient: shutdown 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 >> v1:172.21.15.7:6792/0 conn(0x7f65240a53a0 legacy=0x7f6524137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f65237fe640 1 -- 172.21.15.7:0/1074765078 reap_dead start 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f65237fe640 1 -- 172.21.15.7:0/1074765078 reap_dead start 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 shutdown_connections 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 >> 172.21.15.7:0/1074765078 conn(0x7f65240a9fa0 msgr2=0x7f65240a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 shutdown_connections 2024-05-08T15:47:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.311+0000 7f652952b640 1 -- 172.21.15.7:0/1074765078 wait complete. 2024-05-08T15:47:18.349 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:18.349 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:47:18.349 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:47:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.456+0000 7f2141bbf640 10 monclient: get_monmap_and_config 2024-05-08T15:47:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.456+0000 7f2141bbf640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.456+0000 7f2141bbf640 10 monclient: monmap: 2024-05-08T15:47:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:18.457736+0000 2024-05-08T15:47:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:18.457736+0000 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.455 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.457+0000 7f2141bbf640 1 Processor -- start 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.457+0000 7f2141bbf640 1 -- start start 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.457+0000 7f2141bbf640 10 monclient: init 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 10 monclient: _add_conns ranks=[0,7,3,8,5,4,1,2,6] 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 10 monclient(hunting): picked mon.noname-a con 0x7f213c0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 10 monclient(hunting): picked mon.noname-i con 0x7f213c0a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 10 monclient(hunting): picked mon.noname-g con 0x7f213c13e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c0a1660 con 0x7f213c0a53a0 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c0a28f0 con 0x7f213c13e310 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c0a2bd0 con 0x7f213c0a5780 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 10 monclient(hunting): authenticate will time out at 846.572833s 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213b7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f213c0a53a0 0x7f213c0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58224/0 (socket says 172.21.15.7:58224) 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213b7fe640 1 -- 172.21.15.7:0/3999891370 learned_addr learned my addr 172.21.15.7:0/3999891370 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2877730245 0 0) 0x7f213c0a1660 con 0x7f213c0a53a0 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): my global_id is 4509 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2124003520 con 0x7f213c0a53a0 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 88232430 0 0) 0x7f213c0a28f0 con 0x7f213c13e310 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): my global_id is 74505 2024-05-08T15:47:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c0a1660 con 0x7f213c13e310 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2734725280 0 0) 0x7f213c0a2bd0 con 0x7f213c0a5780 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient(hunting): my global_id is 4444 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c0a28f0 con 0x7f213c0a5780 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 180037906 0 0) 0x7f213c0a1660 con 0x7f213c13e310 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 >> v1:172.21.15.81:6792/0 conn(0x7f213c0a5780 legacy=0x7f213c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 >> v1:172.21.15.7:6789/0 conn(0x7f213c0a53a0 legacy=0x7f213c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 monclient: found mon.noname-g 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c0a6730 con 0x7f213c13e310 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2134001b10 con 0x7f213c13e310 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 5 monclient: authenticate success, global_id 74505 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f213a7fc640 10 monclient: dump: 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.467 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.468 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.458+0000 7f2141bbf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2134001df0 con 0x7f213c13e310 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f213a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f213a7fc640 1 -- 172.21.15.7:0/3999891370 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2134003cd0 con 0x7f213c13e310 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 4 set_mon_vals no callback set 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f213a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f213a7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f213a7fc640 10 monclient: dump: 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.469 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 10 monclient: shutdown 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 1 -- 172.21.15.7:0/3999891370 >> v1:172.21.15.81:6790/0 conn(0x7f213c13e310 legacy=0x7f213c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 1 -- 172.21.15.7:0/3999891370 shutdown_connections 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 1 -- 172.21.15.7:0/3999891370 >> 172.21.15.7:0/3999891370 conn(0x7f213c0a9fa0 msgr2=0x7f213c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 1 -- 172.21.15.7:0/3999891370 shutdown_connections 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.459+0000 7f2141bbf640 1 -- 172.21.15.7:0/3999891370 wait complete. 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.460+0000 7f2141bbf640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.460+0000 7f2141bbf640 10 monclient: monmap: 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.460+0000 7f2141bbf640 1 Processor -- start 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.460+0000 7f2141bbf640 1 -- start start 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.460+0000 7f2141bbf640 10 monclient: init 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 10 monclient: _add_conns ranks=[0,8,5,2,4,1,6,7,3] 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 10 monclient(hunting): picked mon.noname-a con 0x7f213c0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 10 monclient(hunting): picked mon.noname-e con 0x7f213c0a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 10 monclient(hunting): picked mon.noname-h con 0x7f213c13e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c135640 con 0x7f213c0a53a0 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c0a43c0 con 0x7f213c13e310 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f213c1ccff0 con 0x7f213c0a5780 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2141bbf640 10 monclient(hunting): authenticate will time out at 846.575864s 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f213b7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f213c0a53a0 0x7f213c1cc1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58236/0 (socket says 172.21.15.7:58236) 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f213b7fe640 1 -- 172.21.15.7:0/2233174761 learned_addr learned my addr 172.21.15.7:0/2233174761 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479904175 0 0) 0x7f213c1ccff0 con 0x7f213c0a5780 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): my global_id is 4427 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2128003520 con 0x7f213c0a5780 2024-05-08T15:47:18.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3091176249 0 0) 0x7f213c135640 con 0x7f213c0a53a0 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): my global_id is 4518 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c1ccff0 con 0x7f213c0a53a0 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1861051785 0 0) 0x7f213c0a6730 con 0x7f213c13e310 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): my global_id is 4496 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c135640 con 0x7f213c13e310 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2477183560 0 0) 0x7f2128003520 con 0x7f213c0a5780 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f213c0a6730 con 0x7f213c0a5780 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2130001a50 con 0x7f213c0a5780 2024-05-08T15:47:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): dump: 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.475 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3012470644 0 0) 0x7f213c1ccff0 con 0x7f213c0a53a0 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2128003520 con 0x7f213c0a53a0 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f212c001450 con 0x7f213c0a53a0 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): dump: 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.476 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1572021774 0 0) 0x7f213c135640 con 0x7f213c13e310 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f213c1ccff0 con 0x7f213c13e310 2024-05-08T15:47:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2134001ad0 con 0x7f213c13e310 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.461+0000 7f2123fff640 10 monclient(hunting): dump: 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3037900446 0 0) 0x7f213c0a6730 con 0x7f213c0a5780 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.81:6791/0 conn(0x7f213c13e310 legacy=0x7f213c1d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.7:6789/0 conn(0x7f213c0a53a0 legacy=0x7f213c1cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 1 monclient: found mon.e 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f213c137090 con 0x7f213c0a5780 2024-05-08T15:47:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 5 monclient: authenticate success, global_id 4427 2024-05-08T15:47:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 10 monclient: _renew_subs 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f213c1380f0 con 0x7f213c0a5780 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 10 monclient: _renew_subs 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f213c133b50 con 0x7f213c0a5780 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f21300038c0 con 0x7f213c0a5780 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2130003db0 con 0x7f213c0a5780 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2139ffb640 4 set_mon_vals no callback set 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2123fff640 10 monclient: dump: 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.462+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f211c0065c0 con 0x7f211c002860 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f213000f130 con 0x7f213c0a5780 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7f2130045c90 con 0x7f213c0a5780 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3033116932 0 0) 0x7f211c0065c0 con 0x7f211c002860 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 10 monclient(hunting): my global_id is 4456 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f213c0a6730 con 0x7f211c002860 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f213bfff640 10 monclient: get_auth_request con 0x7f212803a470 auth_method 0 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3350485213 0 0) 0x7f213c0a6730 con 0x7f211c002860 2024-05-08T15:47:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f211c0065c0 con 0x7f211c002860 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f212c004b20 con 0x7f211c002860 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 10 monclient: dump: 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2892211743 0 0) 0x7f211c0065c0 con 0x7f211c002860 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.463+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f211c00a150 con 0x7f211c002860 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.464+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f212c004ed0 con 0x7f211c002860 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.464+0000 7f2123fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.464+0000 7f2123fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.464+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.81:6789/0 conn(0x7f211c002860 legacy=0x7f211c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.464+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.81:6789/0 conn(0x7f211c002860 legacy=0x7f211c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.470+0000 7f2141bbf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.470+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f211c009e20 con 0x7f211c0090e0 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3625497884 0 0) 0x7f211c009e20 con 0x7f211c0090e0 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 10 monclient(hunting): my global_id is 4465 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f211c0065c0 con 0x7f211c0090e0 2024-05-08T15:47:18.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1982505914 0 0) 0x7f211c0065c0 con 0x7f211c0090e0 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f211c009e20 con 0x7f211c0090e0 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f212c005890 con 0x7f211c0090e0 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 10 monclient: dump: 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3062131620 0 0) 0x7f211c009e20 con 0x7f211c0090e0 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.471+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f211c00c390 con 0x7f211c0090e0 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 3140339832) 0x7f212c005ba0 con 0x7f211c0090e0 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2123fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2123fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.81:6789/0 conn(0x7f211c0090e0 legacy=0x7f211c009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2123fff640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.81:6789/0 conn(0x7f211c0090e0 legacy=0x7f211c009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 30787, 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:18.490 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 24, 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.491 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.492 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:18.493 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:18.494 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.495 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.497 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:18.498 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:18.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.500 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:18.502 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.504 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.507 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.508 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.509 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:47:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.510 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:18.510 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.81:6812/847627118 conn(0x7f212803a470 legacy=0x7f212803c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 10 monclient: shutdown 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 >> v1:172.21.15.7:6793/0 conn(0x7f213c0a5780 legacy=0x7f213c1cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f213bfff640 1 -- 172.21.15.7:0/2233174761 reap_dead start 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f213bfff640 1 -- 172.21.15.7:0/2233174761 reap_dead start 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 shutdown_connections 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 >> 172.21.15.7:0/2233174761 conn(0x7f213c0a9fa0 msgr2=0x7f213c0a8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 shutdown_connections 2024-05-08T15:47:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.472+0000 7f2141bbf640 1 -- 172.21.15.7:0/2233174761 wait complete. 2024-05-08T15:47:18.512 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:18.512 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:47:18.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.620+0000 7f6aa59ab640 10 monclient: get_monmap_and_config 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.620+0000 7f6aa59ab640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.620+0000 7f6aa59ab640 10 monclient: monmap: 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:18.621609+0000 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:18.621609+0000 2024-05-08T15:47:18.617 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.618 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 1 Processor -- start 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 1 -- start start 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient: init 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient: _add_conns ranks=[8,6,7,5,3,2,4,1,0] 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient(hunting): picked mon.noname-e con 0x7f6aa0137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient(hunting): picked mon.noname-d con 0x7f6aa00a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient(hunting): picked mon.noname-i con 0x7f6aa00a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa00a2c70 con 0x7f6aa00a49a0 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa00a2f50 con 0x7f6aa00a41b0 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa00a1670 con 0x7f6aa0137940 2024-05-08T15:47:18.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.621+0000 7f6aa59ab640 10 monclient(hunting): authenticate will time out at 846.736543s 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9effd640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f6aa0137940 0x7f6aa0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54978/0 (socket says 172.21.15.7:54978) 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a97fff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f6aa00a49a0 0x7f6aa013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:57036/0 (socket says 172.21.15.7:57036) 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9effd640 1 -- 172.21.15.7:0/1240232945 learned_addr learned my addr 172.21.15.7:0/1240232945 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512871372 0 0) 0x7f6aa00a1670 con 0x7f6aa0137940 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): my global_id is 4436 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a8c003520 con 0x7f6aa0137940 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237433620 0 0) 0x7f6aa00a2c70 con 0x7f6aa00a49a0 2024-05-08T15:47:18.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient(hunting): my global_id is 74517 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6aa00a1670 con 0x7f6aa00a49a0 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 853085219 0 0) 0x7f6a8c003520 con 0x7f6aa0137940 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 >> v1:172.21.15.81:6792/0 conn(0x7f6aa00a41b0 legacy=0x7f6aa0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 >> v1:172.21.15.7:6792/0 conn(0x7f6aa00a49a0 legacy=0x7f6aa013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 monclient: found mon.noname-e 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa00a39f0 con 0x7f6aa0137940 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a900017b0 con 0x7f6aa0137940 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6aa59ab640 5 monclient: authenticate success, global_id 4436 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6aa59ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: dump: 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.630 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6aa59ab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6a90001a90 con 0x7f6aa0137940 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6aa59ab640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 1 -- 172.21.15.7:0/1240232945 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a90004be0 con 0x7f6aa0137940 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6aa59ab640 4 set_mon_vals no callback set 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.622+0000 7f6a9e7fc640 10 monclient: dump: 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.632 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 10 monclient: shutdown 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 1 -- 172.21.15.7:0/1240232945 >> v1:172.21.15.7:6793/0 conn(0x7f6aa0137940 legacy=0x7f6aa0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 1 -- 172.21.15.7:0/1240232945 shutdown_connections 2024-05-08T15:47:18.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 1 -- 172.21.15.7:0/1240232945 >> 172.21.15.7:0/1240232945 conn(0x7f6aa00a9fa0 msgr2=0x7f6aa00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 1 -- 172.21.15.7:0/1240232945 shutdown_connections 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 1 -- 172.21.15.7:0/1240232945 wait complete. 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.623+0000 7f6aa59ab640 10 monclient: monmap: 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.634 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 1 Processor -- start 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 1 -- start start 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient: init 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient: _add_conns ranks=[1,6,5,4,8,2,3,0,7] 2024-05-08T15:47:18.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient(hunting): picked mon.noname-f con 0x7f6aa00a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient(hunting): picked mon.noname-d con 0x7f6aa00a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient(hunting): picked mon.noname-h con 0x7f6aa0137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa00a1f30 con 0x7f6aa00a41b0 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa00a0f60 con 0x7f6aa0137940 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa01356d0 con 0x7f6aa00a49a0 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6aa59ab640 10 monclient(hunting): authenticate will time out at 846.739300s 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a97fff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f6aa00a49a0 0x7f6aa0141090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:57050/0 (socket says 172.21.15.7:57050) 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a97fff640 1 -- 172.21.15.7:0/3108625828 learned_addr learned my addr 172.21.15.7:0/3108625828 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3971385848 0 0) 0x7f6aa01356d0 con 0x7f6aa00a49a0 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a9cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a9cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a9cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a9cff9640 10 monclient(hunting): my global_id is 74526 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.624+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a80003520 con 0x7f6aa00a49a0 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3136881041 0 0) 0x7f6a80003520 con 0x7f6aa00a49a0 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6aa01356d0 con 0x7f6aa00a49a0 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a88001460 con 0x7f6aa00a49a0 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): dump: 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.637 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2021593225 0 0) 0x7f6aa00a0f60 con 0x7f6aa0137940 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): my global_id is 4505 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a80003520 con 0x7f6aa0137940 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 125764357 0 0) 0x7f6aa00a1f30 con 0x7f6aa00a41b0 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient(hunting): my global_id is 4474 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6aa00a0f60 con 0x7f6aa00a41b0 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1940533172 0 0) 0x7f6aa01356d0 con 0x7f6aa00a49a0 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.81:6791/0 conn(0x7f6aa0137940 legacy=0x7f6aa01d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.81:6789/0 conn(0x7f6aa00a41b0 legacy=0x7f6aa01cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 monclient: found mon.d 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:18.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa0135f90 con 0x7f6aa00a49a0 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6aa59ab640 5 monclient: authenticate success, global_id 74526 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6aa59ab640 10 monclient: _renew_subs 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6aa59ab640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa01417a0 con 0x7f6aa00a49a0 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6aa59ab640 10 monclient: _renew_subs 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6aa59ab640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f6aa0136410 con 0x7f6aa00a49a0 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6a88004a90 con 0x7f6aa00a49a0 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a88004f80 con 0x7f6aa00a49a0 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9dffb640 4 set_mon_vals no callback set 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.625+0000 7f6a9cff9640 10 monclient: dump: 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.641 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6a88010380 con 0x7f6aa00a49a0 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6aa59ab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6a840065c0 con 0x7f6a84002860 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7f6a88046f00 con 0x7f6aa00a49a0 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9f7fe640 10 monclient: get_auth_request con 0x7f6a80039ca0 auth_method 0 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 384590418 0 0) 0x7f6a840065c0 con 0x7f6a84002860 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 10 monclient(hunting): my global_id is 74514 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.626+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6aa01356d0 con 0x7f6a84002860 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3205646498 0 0) 0x7f6aa01356d0 con 0x7f6a84002860 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a840065c0 con 0x7f6a84002860 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a900011d0 con 0x7f6a84002860 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 10 monclient: dump: 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.643 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 521629625 0 0) 0x7f6a840065c0 con 0x7f6a84002860 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a8400a150 con 0x7f6a84002860 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f6a900017c0 con 0x7f6a84002860 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.81:6790/0 conn(0x7f6a84002860 legacy=0x7f6a84004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.627+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.81:6790/0 conn(0x7f6a84002860 legacy=0x7f6a84004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6aa59ab640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6a84009e40 con 0x7f6a84009100 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777707823 0 0) 0x7f6a84009e40 con 0x7f6a84009100 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 10 monclient(hunting): my global_id is 74523 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a840065c0 con 0x7f6a84009100 2024-05-08T15:47:18.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3823271108 0 0) 0x7f6a840065c0 con 0x7f6a84009100 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a84009e40 con 0x7f6a84009100 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a90005000 con 0x7f6a84009100 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 10 monclient: dump: 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.647 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3794160294 0 0) 0x7f6a84009e40 con 0x7f6a84009100 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.634+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6a8400c3d0 con 0x7f6a84009100 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 2974789312) 0x7f6a90007390 con 0x7f6a84009100 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6a9cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:18.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6a9cff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.81:6790/0 conn(0x7f6a84009100 legacy=0x7f6a84009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6a9cff9640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.81:6790/0 conn(0x7f6a84009100 legacy=0x7f6a84009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 30924, 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:18.649 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 24, 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.650 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.651 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.652 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.653 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.655 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.657 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:18.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.662 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:18.664 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.665 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:18.666 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.81:6812/847627118 conn(0x7f6a80039ca0 legacy=0x7f6a8003c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 10 monclient: shutdown 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 >> v1:172.21.15.7:6792/0 conn(0x7f6aa00a49a0 legacy=0x7f6aa0141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6a9f7fe640 1 -- 172.21.15.7:0/3108625828 reap_dead start 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6a9f7fe640 1 -- 172.21.15.7:0/3108625828 reap_dead start 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 shutdown_connections 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 >> 172.21.15.7:0/3108625828 conn(0x7f6aa00a9fa0 msgr2=0x7f6aa01307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 shutdown_connections 2024-05-08T15:47:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.635+0000 7f6aa59ab640 1 -- 172.21.15.7:0/3108625828 wait complete. 2024-05-08T15:47:18.669 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:18.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:47:18.669 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.776+0000 7f871a645640 10 monclient: get_monmap_and_config 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.776+0000 7f871a645640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.776+0000 7f871a645640 10 monclient: monmap: 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:18.777638+0000 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:18.777638+0000 2024-05-08T15:47:18.773 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.774 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.777+0000 7f871a645640 1 Processor -- start 2024-05-08T15:47:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.777+0000 7f871a645640 1 -- start start 2024-05-08T15:47:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.777+0000 7f871a645640 10 monclient: init 2024-05-08T15:47:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.777+0000 7f871a645640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.777+0000 7f871a645640 10 monclient: _add_conns ranks=[7,1,0,8,2,6,5,3,4] 2024-05-08T15:47:18.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 10 monclient(hunting): picked mon.noname-i con 0x7f8714137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 10 monclient(hunting): picked mon.noname-f con 0x7f87140a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 10 monclient(hunting): picked mon.noname-a con 0x7f87140a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140a2c70 con 0x7f87140a41b0 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140a2f50 con 0x7f87140a49a0 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140a1670 con 0x7f8714137940 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 10 monclient(hunting): authenticate will time out at 846.892764s 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8718be2640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f87140a41b0 0x7f8714140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58248/0 (socket says 172.21.15.7:58248) 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8718be2640 1 -- 172.21.15.7:0/1928241520 learned_addr learned my addr 172.21.15.7:0/1928241520 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139162546 0 0) 0x7f87140a2c70 con 0x7f87140a41b0 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): my global_id is 4527 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86fc003520 con 0x7f87140a41b0 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1437061196 0 0) 0x7f87140a1670 con 0x7f8714137940 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): my global_id is 4462 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87140a2c70 con 0x7f8714137940 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1865800139 0 0) 0x7f87140a2f50 con 0x7f87140a49a0 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient(hunting): my global_id is 4483 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87140a1670 con 0x7f87140a49a0 2024-05-08T15:47:18.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3002629369 0 0) 0x7f86fc003520 con 0x7f87140a41b0 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 >> v1:172.21.15.81:6792/0 conn(0x7f8714137940 legacy=0x7f8714137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 >> v1:172.21.15.81:6789/0 conn(0x7f87140a49a0 legacy=0x7f871413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 monclient: found mon.noname-a 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87140a39f0 con 0x7f87140a41b0 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f870c001b30 con 0x7f87140a41b0 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 5 monclient: authenticate success, global_id 4527 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: dump: 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.787 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f870c001d90 con 0x7f87140a41b0 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 1 -- 172.21.15.7:0/1928241520 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f870c004f60 con 0x7f87140a41b0 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f871a645640 4 set_mon_vals no callback set 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.778+0000 7f8712ffd640 10 monclient: dump: 2024-05-08T15:47:18.788 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.789 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.779+0000 7f871a645640 10 monclient: shutdown 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.779+0000 7f871a645640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.779+0000 7f871a645640 1 -- 172.21.15.7:0/1928241520 >> v1:172.21.15.7:6789/0 conn(0x7f87140a41b0 legacy=0x7f8714140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.779+0000 7f871a645640 1 -- 172.21.15.7:0/1928241520 shutdown_connections 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.779+0000 7f871a645640 1 -- 172.21.15.7:0/1928241520 >> 172.21.15.7:0/1928241520 conn(0x7f87140a9fa0 msgr2=0x7f87140aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.779+0000 7f871a645640 1 -- 172.21.15.7:0/1928241520 shutdown_connections 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.779+0000 7f871a645640 1 -- 172.21.15.7:0/1928241520 wait complete. 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 10 monclient: monmap: 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.790 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 1 Processor -- start 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 1 -- start start 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 10 monclient: init 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 10 monclient: _add_conns ranks=[8,5,4,7,1,0,3,6,2] 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.780+0000 7f871a645640 10 monclient(hunting): picked mon.noname-e con 0x7f87140a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 10 monclient(hunting): picked mon.noname-h con 0x7f87140a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 10 monclient(hunting): picked mon.noname-c con 0x7f8714137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140a1f30 con 0x7f8714137940 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87140a0f60 con 0x7f87140a49a0 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87141340a0 con 0x7f87140a41b0 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 10 monclient(hunting): authenticate will time out at 846.895726s 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8713fff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f87140a41b0 0x7f8714136120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:54992/0 (socket says 172.21.15.7:54992) 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8713fff640 1 -- 172.21.15.7:0/3651158897 learned_addr learned my addr 172.21.15.7:0/3651158897 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3404273142 0 0) 0x7f87140a1f30 con 0x7f8714137940 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): my global_id is 4432 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8700003520 con 0x7f8714137940 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2817670623 0 0) 0x7f87140a0f60 con 0x7f87140a49a0 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): my global_id is 4514 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87140a1f30 con 0x7f87140a49a0 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1567349189 0 0) 0x7f87141340a0 con 0x7f87140a41b0 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): my global_id is 4445 2024-05-08T15:47:18.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87140a0f60 con 0x7f87140a41b0 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2316671017 0 0) 0x7f87140a0f60 con 0x7f87140a41b0 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87141340a0 con 0x7f87140a41b0 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8704001a30 con 0x7f87140a41b0 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.794 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1505801322 0 0) 0x7f87140a1f30 con 0x7f87140a49a0 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87140a0f60 con 0x7f87140a49a0 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4110199645 0 0) 0x7f8700003520 con 0x7f8714137940 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87140a1f30 con 0x7f8714137940 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8708001950 con 0x7f87140a49a0 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.796 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f870c004ea0 con 0x7f8714137940 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.797 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1044999870 0 0) 0x7f87141340a0 con 0x7f87140a41b0 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.81:6791/0 conn(0x7f87140a49a0 legacy=0x7f87141d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.7:6791/0 conn(0x7f8714137940 legacy=0x7f87141d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 monclient: found mon.e 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8714135d00 con 0x7f87140a41b0 2024-05-08T15:47:18.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f8710ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.781+0000 7f871a645640 5 monclient: authenticate success, global_id 4445 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 10 monclient: _renew_subs 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8714135460 con 0x7f87140a41b0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 10 monclient: _renew_subs 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f8714134720 con 0x7f87140a41b0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8704001d10 con 0x7f87140a41b0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f8710ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8704003c00 con 0x7f87140a41b0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f8710ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f8710ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f87127fc640 4 set_mon_vals no callback set 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f8710ff9640 10 monclient: dump: 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.800 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f8704003da0 con 0x7f87140a41b0 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.782+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f86f40065c0 con 0x7f86f4002860 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7f8704046cb0 con 0x7f87140a41b0 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8718be2640 10 monclient: get_auth_request con 0x7f870003a3f0 auth_method 0 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4060340153 0 0) 0x7f86f40065c0 con 0x7f86f4002860 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 10 monclient(hunting): my global_id is 4523 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87141340a0 con 0x7f86f4002860 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3737031773 0 0) 0x7f87141340a0 con 0x7f86f4002860 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f86f40065c0 con 0x7f86f4002860 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8708003bc0 con 0x7f86f4002860 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 10 monclient: dump: 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.802 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4251918178 0 0) 0x7f86f40065c0 con 0x7f86f4002860 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.783+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f86f400a150 con 0x7f86f4002860 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.784+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f8708003f50 con 0x7f86f4002860 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.784+0000 7f8710ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.784+0000 7f8710ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.784+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.81:6791/0 conn(0x7f86f4002860 legacy=0x7f86f4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.784+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.81:6791/0 conn(0x7f86f4002860 legacy=0x7f86f4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.790+0000 7f871a645640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.790+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f86f4009e20 con 0x7f86f40090e0 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1583179707 0 0) 0x7f86f4009e20 con 0x7f86f40090e0 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 10 monclient(hunting): my global_id is 4532 2024-05-08T15:47:18.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86f40065c0 con 0x7f86f40090e0 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2619109962 0 0) 0x7f86f40065c0 con 0x7f86f40090e0 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f86f4009e20 con 0x7f86f40090e0 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8708004730 con 0x7f86f40090e0 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 10 monclient: dump: 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.806 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2725559930 0 0) 0x7f86f4009e20 con 0x7f86f40090e0 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.791+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f86f400c390 con 0x7f86f40090e0 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 3953401538) 0x7f8708005b60 con 0x7f86f40090e0 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f8710ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:18.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f8710ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.81:6791/0 conn(0x7f86f40090e0 legacy=0x7f86f4009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f8710ff9640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.81:6791/0 conn(0x7f86f40090e0 legacy=0x7f86f4009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 31097, 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:18.808 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 24, 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:18.809 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.810 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:18.811 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:18.812 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:18.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.814 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:18.820 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:18.821 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.823 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:18.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:18.825 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:18.826 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:18.827 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:18.827 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.81:6812/847627118 conn(0x7f870003a3f0 legacy=0x7f870003c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 10 monclient: shutdown 2024-05-08T15:47:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 >> v1:172.21.15.7:6793/0 conn(0x7f87140a41b0 legacy=0x7f8714136120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f8718be2640 1 -- 172.21.15.7:0/3651158897 reap_dead start 2024-05-08T15:47:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f8718be2640 1 -- 172.21.15.7:0/3651158897 reap_dead start 2024-05-08T15:47:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 shutdown_connections 2024-05-08T15:47:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 >> 172.21.15.7:0/3651158897 conn(0x7f87140a9fa0 msgr2=0x7f8714130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 shutdown_connections 2024-05-08T15:47:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.792+0000 7f871a645640 1 -- 172.21.15.7:0/3651158897 wait complete. 2024-05-08T15:47:18.829 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:18.829 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:47:18.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.938+0000 7fdf933b6640 10 monclient: get_monmap_and_config 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.938+0000 7fdf933b6640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.938+0000 7fdf933b6640 10 monclient: monmap: 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:18.939595+0000 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:18.939595+0000 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.935 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.936 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.936 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.936 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.936 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.936 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.936 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.939+0000 7fdf933b6640 1 Processor -- start 2024-05-08T15:47:18.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.939+0000 7fdf933b6640 1 -- start start 2024-05-08T15:47:18.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.939+0000 7fdf933b6640 10 monclient: init 2024-05-08T15:47:18.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient: _add_conns ranks=[8,3,4,7,5,0,2,1,6] 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient(hunting): picked mon.noname-e con 0x7fdf8c137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient(hunting): picked mon.noname-g con 0x7fdf8c0a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient(hunting): picked mon.noname-c con 0x7fdf8c0a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c0a2c70 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c0a2f50 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c0a1670 con 0x7fdf8c137940 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf91152640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fdf8c137940 0x7fdf8c137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55002/0 (socket says 172.21.15.7:55002) 2024-05-08T15:47:18.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient(hunting): authenticate will time out at 847.054823s 2024-05-08T15:47:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf91152640 1 -- 172.21.15.7:0/4165984688 learned_addr learned my addr 172.21.15.7:0/4165984688 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2082775755 0 0) 0x7fdf8c0a1670 con 0x7fdf8c137940 2024-05-08T15:47:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): my global_id is 4454 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf78003520 con 0x7fdf8c137940 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906926387 0 0) 0x7fdf8c0a2f50 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): my global_id is 4441 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf8c0a1670 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099087353 0 0) 0x7fdf8c0a2c70 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient(hunting): my global_id is 74532 2024-05-08T15:47:18.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf8c0a2f50 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3312600518 0 0) 0x7fdf8c0a1670 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 >> v1:172.21.15.7:6793/0 conn(0x7fdf8c137940 legacy=0x7fdf8c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 >> v1:172.21.15.81:6790/0 conn(0x7fdf8c0a49a0 legacy=0x7fdf8c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 monclient: found mon.noname-c 2024-05-08T15:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c0a39f0 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf84001b10 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 5 monclient: authenticate success, global_id 4441 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: dump: 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.950 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.951 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdf84001e70 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 1 -- 172.21.15.7:0/4165984688 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf84003cd0 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf933b6640 4 set_mon_vals no callback set 2024-05-08T15:47:18.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.940+0000 7fdf83fff640 10 monclient: dump: 2024-05-08T15:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.953 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.954 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.955 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.941+0000 7fdf933b6640 10 monclient: shutdown 2024-05-08T15:47:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.941+0000 7fdf933b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.941+0000 7fdf933b6640 1 -- 172.21.15.7:0/4165984688 >> v1:172.21.15.7:6791/0 conn(0x7fdf8c0a41b0 legacy=0x7fdf8c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.941+0000 7fdf933b6640 1 -- 172.21.15.7:0/4165984688 shutdown_connections 2024-05-08T15:47:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.941+0000 7fdf933b6640 1 -- 172.21.15.7:0/4165984688 >> 172.21.15.7:0/4165984688 conn(0x7fdf8c0a9fa0 msgr2=0x7fdf8c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:18.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.941+0000 7fdf933b6640 1 -- 172.21.15.7:0/4165984688 shutdown_connections 2024-05-08T15:47:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.941+0000 7fdf933b6640 1 -- 172.21.15.7:0/4165984688 wait complete. 2024-05-08T15:47:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient: build_initial_monmap 2024-05-08T15:47:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient: monmap: 2024-05-08T15:47:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:18.957 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:18.958 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:18.959 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:18.960 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 1 Processor -- start 2024-05-08T15:47:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 1 -- start start 2024-05-08T15:47:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient: init 2024-05-08T15:47:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:18.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient: _add_conns ranks=[3,7,5,2,6,1,8,4,0] 2024-05-08T15:47:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient(hunting): picked mon.noname-g con 0x7fdf8c0a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient(hunting): picked mon.noname-i con 0x7fdf8c0a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient(hunting): picked mon.noname-h con 0x7fdf8c137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c135de0 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c0a1f30 con 0x7fdf8c137940 2024-05-08T15:47:18.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf8c0a0f60 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.942+0000 7fdf933b6640 10 monclient(hunting): authenticate will time out at 847.057557s 2024-05-08T15:47:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf91152640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fdf8c0a41b0 0x7fdf8c140e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:37056/0 (socket says 172.21.15.7:37056) 2024-05-08T15:47:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf90951640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7fdf8c0a49a0 0x7fdf8c1d14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:43020/0 (socket says 172.21.15.7:43020) 2024-05-08T15:47:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf91152640 1 -- 172.21.15.7:0/2365958201 learned_addr learned my addr 172.21.15.7:0/2365958201 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:18.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf91953640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fdf8c137940 0x7fdf8c1d4be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:33384/0 (socket says 172.21.15.7:33384) 2024-05-08T15:47:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232226626 0 0) 0x7fdf8c0a39f0 con 0x7fdf8c137940 2024-05-08T15:47:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): my global_id is 4541 2024-05-08T15:47:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf70003520 con 0x7fdf8c137940 2024-05-08T15:47:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009767497 0 0) 0x7fdf8c135de0 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): my global_id is 74541 2024-05-08T15:47:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf8c0a39f0 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2204256870 0 0) 0x7fdf8c0a0f60 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): my global_id is 4471 2024-05-08T15:47:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf8c135de0 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3502240677 0 0) 0x7fdf70003520 con 0x7fdf8c137940 2024-05-08T15:47:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf8c0a0f60 con 0x7fdf8c137940 2024-05-08T15:47:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf84001ad0 con 0x7fdf8c137940 2024-05-08T15:47:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:47:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.968 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.969 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.969 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 951723352 0 0) 0x7fdf8c0a39f0 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf70003520 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf88001970 con 0x7fdf8c0a41b0 2024-05-08T15:47:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.971 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.972 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.973 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 843207182 0 0) 0x7fdf8c135de0 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf8c0a39f0 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf7c001290 con 0x7fdf8c0a49a0 2024-05-08T15:47:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:18.974 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.975 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.976 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.977 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.977 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.977 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2860961017 0 0) 0x7fdf8c0a0f60 con 0x7fdf8c137940 2024-05-08T15:47:18.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6792/0 conn(0x7fdf8c0a49a0 legacy=0x7fdf8c1d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6790/0 conn(0x7fdf8c0a41b0 legacy=0x7fdf8c140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:18.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 monclient: found mon.h 2024-05-08T15:47:18.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:18.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:18.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.943+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c136510 con 0x7fdf8c137940 2024-05-08T15:47:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 5 monclient: authenticate success, global_id 4541 2024-05-08T15:47:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 10 monclient: _renew_subs 2024-05-08T15:47:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c1398c0 con 0x7fdf8c137940 2024-05-08T15:47:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdf84002020 con 0x7fdf8c137940 2024-05-08T15:47:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf84004620 con 0x7fdf8c137940 2024-05-08T15:47:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf837fe640 4 set_mon_vals no callback set 2024-05-08T15:47:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf81ffb640 10 monclient: dump: 2024-05-08T15:47:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.984 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 10 monclient: _renew_subs 2024-05-08T15:47:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fdf8c1c8ec0 con 0x7fdf8c137940 2024-05-08T15:47:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.944+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdf8c0753d0 con 0x7fdf8c141560 2024-05-08T15:47:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fdf84004840 con 0x7fdf8c137940 2024-05-08T15:47:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.5 v1:172.21.15.81:6791/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7fdf84047290 con 0x7fdf8c137940 2024-05-08T15:47:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999573354 0 0) 0x7fdf8c0753d0 con 0x7fdf8c141560 2024-05-08T15:47:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 10 monclient(hunting): my global_id is 4480 2024-05-08T15:47:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf8c0a0f60 con 0x7fdf8c141560 2024-05-08T15:47:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf90951640 10 monclient: get_auth_request con 0x7fdf7003a460 auth_method 0 2024-05-08T15:47:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2209990406 0 0) 0x7fdf8c0a0f60 con 0x7fdf8c141560 2024-05-08T15:47:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf8c0753d0 con 0x7fdf8c141560 2024-05-08T15:47:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf88004b20 con 0x7fdf8c141560 2024-05-08T15:47:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 10 monclient: dump: 2024-05-08T15:47:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:18.991 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 297347912 0 0) 0x7fdf8c0753d0 con 0x7fdf8c141560 2024-05-08T15:47:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.945+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdf8c141d00 con 0x7fdf8c141560 2024-05-08T15:47:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.946+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fdf88005020 con 0x7fdf8c141560 2024-05-08T15:47:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.946+0000 7fdf81ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.946+0000 7fdf81ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.946+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6792/0 conn(0x7fdf8c141560 legacy=0x7fdf8c09da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.946+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6792/0 conn(0x7fdf8c141560 legacy=0x7fdf8c09da60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.952+0000 7fdf933b6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.952+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdf8c06b2f0 con 0x7fdf8c02d050 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2625024266 0 0) 0x7fdf8c06b2f0 con 0x7fdf8c02d050 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 10 monclient(hunting): my global_id is 4489 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf8c0753d0 con 0x7fdf8c02d050 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3751449645 0 0) 0x7fdf8c0753d0 con 0x7fdf8c02d050 2024-05-08T15:47:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdf8c06b2f0 con 0x7fdf8c02d050 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdf88005780 con 0x7fdf8c02d050 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 10 monclient: dump: 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 95859341 0 0) 0x7fdf8c06b2f0 con 0x7fdf8c02d050 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.953+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdf8c02d430 con 0x7fdf8c02d050 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 3139068863) 0x7fdf88006b00 con 0x7fdf8c02d050 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf81ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf81ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6792/0 conn(0x7fdf8c02d050 legacy=0x7fdf8c09e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf81ffb640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6792/0 conn(0x7fdf8c02d050 legacy=0x7fdf8c09e170 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 10, 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 31249, 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:47:19.002 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 25, 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:19.003 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:19.004 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:19.005 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:19.006 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:19.007 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:47:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:19.020 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6812/847627118 conn(0x7fdf7003a460 legacy=0x7fdf7003c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 10 monclient: shutdown 2024-05-08T15:47:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 >> v1:172.21.15.81:6791/0 conn(0x7fdf8c137940 legacy=0x7fdf8c1d4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf91953640 1 -- 172.21.15.7:0/2365958201 reap_dead start 2024-05-08T15:47:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf91953640 1 -- 172.21.15.7:0/2365958201 reap_dead start 2024-05-08T15:47:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 shutdown_connections 2024-05-08T15:47:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 >> 172.21.15.7:0/2365958201 conn(0x7fdf8c0a9fa0 msgr2=0x7fdf8c0a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 shutdown_connections 2024-05-08T15:47:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:18.954+0000 7fdf933b6640 1 -- 172.21.15.7:0/2365958201 wait complete. 2024-05-08T15:47:19.022 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:47:19.022 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i', 'e', 'b', 'a'] 2024-05-08T15:47:19.022 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T15:47:19.022 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-05-08T15:47:19.023 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-05-08T15:47:19.023 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-05-08T15:47:19.023 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:47:19.035 INFO:tasks.ceph.mon.i:Stopped 2024-05-08T15:47:19.036 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-05-08T15:47:19.036 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-05-08T15:47:19.036 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-05-08T15:47:19.036 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:47:19.046 INFO:tasks.ceph.mon.e:Stopped 2024-05-08T15:47:19.046 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-05-08T15:47:19.046 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-08T15:47:19.047 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T15:47:19.047 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:47:19.098 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T15:47:19.098 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-05-08T15:47:19.098 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T15:47:19.098 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T15:47:19.099 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:47:19.149 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T15:47:19.149 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 5 2024-05-08T15:47:19.150 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:47:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.295+0000 7fb68b693640 10 monclient: get_monmap_and_config 2024-05-08T15:47:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.295+0000 7fb68b693640 10 monclient: build_initial_monmap 2024-05-08T15:47:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.295+0000 7fb68b693640 10 monclient: monmap: 2024-05-08T15:47:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:19.291 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:19.296033+0000 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:19.296033+0000 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:19.292 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.295+0000 7fb68b693640 1 Processor -- start 2024-05-08T15:47:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.295+0000 7fb68b693640 1 -- start start 2024-05-08T15:47:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.295+0000 7fb68b693640 10 monclient: init 2024-05-08T15:47:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 10 monclient: _add_conns ranks=[3,7,6,2,1,5,0,4,8] 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 10 monclient(hunting): picked mon.noname-g con 0x7fb6840a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 10 monclient(hunting): picked mon.noname-i con 0x7fb6840a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 10 monclient(hunting): picked mon.noname-d con 0x7fb68413e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6840a1660 con 0x7fb6840a53a0 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6840a28f0 con 0x7fb68413e310 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6840a2bd0 con 0x7fb6840a5780 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb68b693640 10 monclient(hunting): authenticate will time out at 847.411039s 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb688c2e640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7fb6840a5780 legacy=0x7fb68413cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb689c30640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fb68413e310 0x7fb684140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:57054/0 (socket says 172.21.15.7:57054) 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb689c30640 1 -- 172.21.15.7:0/3111779543 learned_addr learned my addr 172.21.15.7:0/3111779543 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2888952497 0 0) 0x7fb6840a28f0 con 0x7fb68413e310 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): my global_id is 74535 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb670003520 con 0x7fb68413e310 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 977768170 0 0) 0x7fb6840a1660 con 0x7fb6840a53a0 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 10 monclient(hunting): my global_id is 74550 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.296+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6840a28f0 con 0x7fb6840a53a0 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1391904286 0 0) 0x7fb670003520 con 0x7fb68413e310 2024-05-08T15:47:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 >> v1:172.21.15.81:6792/0 conn(0x7fb6840a5780 legacy=0x7fb68413cbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 >> v1:172.21.15.81:6790/0 conn(0x7fb6840a53a0 legacy=0x7fb6840a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 monclient: found mon.noname-d 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6840a6730 con 0x7fb68413e310 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: _finish_auth 0 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb67c001b30 con 0x7fb68413e310 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: dump: 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:19.304 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 5 monclient: authenticate success, global_id 74535 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb67c001e50 con 0x7fb68413e310 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 1 -- 172.21.15.7:0/3111779543 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb67c004f60 con 0x7fb68413e310 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb67bfff640 10 monclient: dump: 2024-05-08T15:47:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 4 set_mon_vals no callback set 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 10 monclient: shutdown 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 1 -- 172.21.15.7:0/3111779543 >> v1:172.21.15.7:6792/0 conn(0x7fb68413e310 legacy=0x7fb684140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 1 -- 172.21.15.7:0/3111779543 shutdown_connections 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 1 -- 172.21.15.7:0/3111779543 >> 172.21.15.7:0/3111779543 conn(0x7fb6840a9fa0 msgr2=0x7fb6840aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 1 -- 172.21.15.7:0/3111779543 shutdown_connections 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.297+0000 7fb68b693640 1 -- 172.21.15.7:0/3111779543 wait complete. 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.298+0000 7fb68b693640 10 monclient: build_initial_monmap 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.298+0000 7fb68b693640 10 monclient: monmap: 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:19.308 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.298+0000 7fb68b693640 1 Processor -- start 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.298+0000 7fb68b693640 1 -- start start 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.298+0000 7fb68b693640 10 monclient: init 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 10 monclient: _add_conns ranks=[5,0,8,3,7,1,4,2,6] 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 10 monclient(hunting): picked mon.noname-h con 0x7fb6840a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 10 monclient(hunting): picked mon.noname-a con 0x7fb6840a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 10 monclient(hunting): picked mon.noname-e con 0x7fb68413e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb684134700 con 0x7fb6840a5780 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb688c2e640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7fb6840a5780 legacy=0x7fb684137f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb684136150 con 0x7fb6840a53a0 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb689c30640 1 -- >> v1:172.21.15.7:6793/0 conn(0x7fb68413e310 legacy=0x7fb6841d4610 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb684136430 con 0x7fb68413e310 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:19.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68b693640 10 monclient(hunting): authenticate will time out at 847.414117s 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68942f640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fb6840a53a0 0x7fb6841cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:33404/0 (socket says 172.21.15.7:33404) 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb68942f640 1 -- 172.21.15.7:0/1877067926 learned_addr learned my addr 172.21.15.7:0/1877067926 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82375110 0 0) 0x7fb684136150 con 0x7fb6840a53a0 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb679ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb679ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb679ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb679ffb640 10 monclient(hunting): my global_id is 4550 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.299+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb668003520 con 0x7fb6840a53a0 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2705113553 0 0) 0x7fb668003520 con 0x7fb6840a53a0 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb684136150 con 0x7fb6840a53a0 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb680001ab0 con 0x7fb6840a53a0 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:47:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient(hunting): dump: 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3241839782 0 0) 0x7fb684136150 con 0x7fb6840a53a0 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 >> v1:172.21.15.7:6793/0 conn(0x7fb68413e310 legacy=0x7fb6841d4610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 >> v1:172.21.15.7:6789/0 conn(0x7fb6840a5780 legacy=0x7fb684137f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 monclient: found mon.h 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb684138630 con 0x7fb6840a53a0 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb68b693640 5 monclient: authenticate success, global_id 4550 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb68b693640 10 monclient: _renew_subs 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb68b693640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 --> v1:172.21.15.81:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb684135380 con 0x7fb6840a53a0 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb6800038c0 con 0x7fb6840a53a0 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb680003e30 con 0x7fb6840a53a0 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb679ffb640 10 monclient: dump: 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:19.313 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb67b7fe640 4 set_mon_vals no callback set 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb68b693640 10 monclient: _renew_subs 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb68b693640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:19.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.300+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 --> v1:172.21.15.81:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fb6841d5d30 con 0x7fb6840a53a0 2024-05-08T15:47:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.301+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fb680004050 con 0x7fb6840a53a0 2024-05-08T15:47:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.301+0000 7fb68b693640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.301+0000 7fb68b693640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.301+0000 7fb68b693640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.301+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb684075820 con 0x7fb6840a53a0 2024-05-08T15:47:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.301+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7fb680047060 con 0x7fb6840a53a0 2024-05-08T15:47:19.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.301+0000 7fb688c2e640 10 monclient: get_auth_request con 0x7fb668039a40 auth_method 0 2024-05-08T15:47:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.307+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fb680016070 con 0x7fb6840a53a0 2024-05-08T15:47:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.307+0000 7fb679ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:19.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.307+0000 7fb679ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:19.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.560+0000 7fb68b693640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:47:19.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.560+0000 7fb68b693640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:19.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.560+0000 7fb68b693640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:19.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.560+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb68406b380 con 0x7fb6840a53a0 2024-05-08T15:47:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.560+0000 7fb679ffb640 1 -- 172.21.15.7:0/1877067926 <== mon.5 v1:172.21.15.81:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2703 (unknown 254746657 0 2687567846) 0x7fb680015970 con 0x7fb6840a53a0 2024-05-08T15:47:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.560+0000 7fb679ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:19.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.560+0000 7fb679ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:19.557 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":10,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 >> v1:172.21.15.81:6812/847627118 conn(0x7fb668039a40 legacy=0x7fb66803bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 10 monclient: shutdown 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 >> v1:172.21.15.81:6791/0 conn(0x7fb6840a53a0 legacy=0x7fb6841cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 shutdown_connections 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 >> 172.21.15.7:0/1877067926 conn(0x7fb6840a9fa0 msgr2=0x7fb6840a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 shutdown_connections 2024-05-08T15:47:19.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:19.562+0000 7fb68b693640 1 -- 172.21.15.7:0/1877067926 wait complete. 2024-05-08T15:47:22.571 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.689+0000 7ff3ee433640 10 monclient: get_monmap_and_config 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.689+0000 7ff3ee433640 10 monclient: build_initial_monmap 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.689+0000 7ff3ee433640 10 monclient: monmap: 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:22.690625+0000 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:22.690625+0000 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:22.686 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:22.687 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.690+0000 7ff3ee433640 1 Processor -- start 2024-05-08T15:47:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.690+0000 7ff3ee433640 1 -- start start 2024-05-08T15:47:22.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.690+0000 7ff3ee433640 10 monclient: init 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.690+0000 7ff3ee433640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.690+0000 7ff3ee433640 10 monclient: _add_conns ranks=[2,5,0,7,8,3,4,1,6] 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.690+0000 7ff3ee433640 10 monclient(hunting): picked mon.noname-b con 0x7ff3e80a1580 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 10 monclient(hunting): picked mon.noname-h con 0x7ff3e80a2070 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 10 monclient(hunting): picked mon.noname-a con 0x7ff3e8072690 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e8072aa0 con 0x7ff3e8072690 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e8070160 con 0x7ff3e80a1580 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e8071410 con 0x7ff3e80a2070 2024-05-08T15:47:22.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 10 monclient(hunting): authenticate will time out at 850.805672s 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ec9d0640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7ff3e8072690 legacy=0x7ff3e813d180 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e7fff640 1 -- >> v1:172.21.15.7:6790/0 conn(0x7ff3e80a1580 legacy=0x7ff3e80a1960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e77fe640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7ff3e80a2070 0x7ff3e8139a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:33420/0 (socket says 172.21.15.7:33420) 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e77fe640 1 -- 172.21.15.7:0/3070095346 learned_addr learned my addr 172.21.15.7:0/3070095346 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4205701935 0 0) 0x7ff3e8071410 con 0x7ff3e80a2070 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:22.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient(hunting): my global_id is 4559 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3d0003520 con 0x7ff3e80a2070 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4152265641 0 0) 0x7ff3d0003520 con 0x7ff3e80a2070 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 >> v1:172.21.15.7:6790/0 conn(0x7ff3e80a1580 legacy=0x7ff3e80a1960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 >> v1:172.21.15.7:6789/0 conn(0x7ff3e8072690 legacy=0x7ff3e813d180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 monclient: found mon.noname-h 2024-05-08T15:47:22.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e806fcd0 con 0x7ff3e80a2070 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff3dc0017b0 con 0x7ff3e80a2070 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 5 monclient: authenticate success, global_id 4559 2024-05-08T15:47:22.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3ee433640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.691+0000 7ff3e6ffd640 10 monclient: dump: 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:22.703 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:22.704 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff3dc001aa0 con 0x7ff3e80a2070 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3e6ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3e6ffd640 1 -- 172.21.15.7:0/3070095346 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff3dc004be0 con 0x7ff3e80a2070 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 4 set_mon_vals no callback set 2024-05-08T15:47:22.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3e6ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3e6ffd640 10 monclient: dump: 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:22.706 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 10 monclient: shutdown 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 1 -- 172.21.15.7:0/3070095346 >> v1:172.21.15.81:6791/0 conn(0x7ff3e80a2070 legacy=0x7ff3e8139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 1 -- 172.21.15.7:0/3070095346 shutdown_connections 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 1 -- 172.21.15.7:0/3070095346 >> 172.21.15.7:0/3070095346 conn(0x7ff3e80a6280 msgr2=0x7ff3e80a2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 1 -- 172.21.15.7:0/3070095346 shutdown_connections 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.692+0000 7ff3ee433640 1 -- 172.21.15.7:0/3070095346 wait complete. 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.693+0000 7ff3ee433640 10 monclient: build_initial_monmap 2024-05-08T15:47:22.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.693+0000 7ff3ee433640 10 monclient: monmap: 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:22.708 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.693+0000 7ff3ee433640 1 Processor -- start 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.693+0000 7ff3ee433640 1 -- start start 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.693+0000 7ff3ee433640 10 monclient: init 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 10 monclient: _add_conns ranks=[5,6,4,3,0,2,1,7,8] 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 10 monclient(hunting): picked mon.noname-h con 0x7ff3e8072690 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 10 monclient(hunting): picked mon.noname-d con 0x7ff3e80a1580 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 10 monclient(hunting): picked mon.noname-c con 0x7ff3e80a2070 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e8122140 con 0x7ff3e80a2070 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e8120bd0 con 0x7ff3e8072690 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3e811fe50 con 0x7ff3e80a1580 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ee433640 10 monclient(hunting): authenticate will time out at 850.809131s 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e77fe640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7ff3e80a1580 0x7ff3e81ccf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:57070/0 (socket says 172.21.15.7:57070) 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e77fe640 1 -- 172.21.15.7:0/1085182559 learned_addr learned my addr 172.21.15.7:0/1085182559 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3ec9d0640 1 --1- 172.21.15.7:0/1085182559 >> v1:172.21.15.7:6791/0 conn(0x7ff3e80a2070 0x7ff3e81d0640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:41188/0 (socket says 172.21.15.7:41188) 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2965334480 0 0) 0x7ff3e811fe50 con 0x7ff3e80a1580 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): my global_id is 74544 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3d4003520 con 0x7ff3e80a1580 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 396722405 0 0) 0x7ff3e8122140 con 0x7ff3e80a2070 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 10 monclient(hunting): my global_id is 4450 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.694+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3e811fe50 con 0x7ff3e80a2070 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1753404221 0 0) 0x7ff3e8120bd0 con 0x7ff3e8072690 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): my global_id is 4568 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3e8122140 con 0x7ff3e8072690 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1284523085 0 0) 0x7ff3d4003520 con 0x7ff3e80a1580 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff3e8120bd0 con 0x7ff3e80a1580 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff3dc001890 con 0x7ff3e80a1580 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:22.712 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2521456142 0 0) 0x7ff3e811fe50 con 0x7ff3e80a2070 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff3d4003520 con 0x7ff3e80a2070 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff3e0004450 con 0x7ff3e80a2070 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3020079530 0 0) 0x7ff3e8122140 con 0x7ff3e8072690 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff3e811fe50 con 0x7ff3e8072690 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff3d8004260 con 0x7ff3e8072690 2024-05-08T15:47:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient(hunting): dump: 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3577924241 0 0) 0x7ff3e8120bd0 con 0x7ff3e80a1580 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 >> v1:172.21.15.81:6791/0 conn(0x7ff3e8072690 legacy=0x7ff3e8134780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 >> v1:172.21.15.7:6791/0 conn(0x7ff3e80a2070 legacy=0x7ff3e81d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 monclient: found mon.d 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e8121e60 con 0x7ff3e80a1580 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: _finish_auth 0 2024-05-08T15:47:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3ee433640 5 monclient: authenticate success, global_id 74544 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3ee433640 10 monclient: _renew_subs 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3ee433640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e81c4920 con 0x7ff3e80a1580 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3ee433640 10 monclient: _renew_subs 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3ee433640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7ff3e8134e90 con 0x7ff3e80a1580 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff3dc001aa0 con 0x7ff3e80a1580 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff3dc005110 con 0x7ff3e80a1580 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e67fc640 4 set_mon_vals no callback set 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.695+0000 7ff3e4ff9640 10 monclient: dump: 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:22.718 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.696+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7ff3dc010490 con 0x7ff3e80a1580 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.696+0000 7ff3ee433640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.696+0000 7ff3ee433640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.696+0000 7ff3ee433640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.696+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c8005080 con 0x7ff3e80a1580 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.697+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7ff3dc046e80 con 0x7ff3e80a1580 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.697+0000 7ff3e7fff640 10 monclient: get_auth_request con 0x7ff3d403a3c0 auth_method 0 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.701+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7ff3dc015f70 con 0x7ff3e80a1580 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.701+0000 7ff3e4ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:22.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.701+0000 7ff3e4ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.957+0000 7ff3ee433640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.957+0000 7ff3ee433640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.957+0000 7ff3ee433640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.957+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff3c8005370 con 0x7ff3e80a1580 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.957+0000 7ff3e4ff9640 1 -- 172.21.15.7:0/1085182559 <== mon.6 v1:172.21.15.7:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2703 (unknown 254746657 0 901681633) 0x7ff3dc019bc0 con 0x7ff3e80a1580 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.957+0000 7ff3e4ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.957+0000 7ff3e4ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:22.954 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":10,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"a","quorum_age":29,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 >> v1:172.21.15.81:6812/847627118 conn(0x7ff3d403a3c0 legacy=0x7ff3d403c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 10 monclient: shutdown 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 >> v1:172.21.15.7:6792/0 conn(0x7ff3e80a1580 legacy=0x7ff3e81ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 shutdown_connections 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 >> 172.21.15.7:0/1085182559 conn(0x7ff3e80a6280 msgr2=0x7ff3e809f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 shutdown_connections 2024-05-08T15:47:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:22.959+0000 7ff3ee433640 1 -- 172.21.15.7:0/1085182559 wait complete. 2024-05-08T15:47:25.967 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:47:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.085+0000 7f2891874640 10 monclient: get_monmap_and_config 2024-05-08T15:47:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.085+0000 7f2891874640 10 monclient: build_initial_monmap 2024-05-08T15:47:26.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.085+0000 7f2891874640 10 monclient: monmap: 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:26.086915+0000 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:26.086915+0000 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:26.083 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:26.084 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:26.084 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:26.084 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:26.084 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:26.084 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:26.084 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.086+0000 7f2891874640 1 Processor -- start 2024-05-08T15:47:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.086+0000 7f2891874640 1 -- start start 2024-05-08T15:47:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.086+0000 7f2891874640 10 monclient: init 2024-05-08T15:47:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 10 monclient: _add_conns ranks=[1,4,5,0,8,7,6,3,2] 2024-05-08T15:47:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 10 monclient(hunting): picked mon.noname-f con 0x7f288c137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:26.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 10 monclient(hunting): picked mon.noname-c con 0x7f288c0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 10 monclient(hunting): picked mon.noname-h con 0x7f288c0a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a2c70 con 0x7f288c137940 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a2f50 con 0x7f288c0a49a0 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a1670 con 0x7f288c0a41b0 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f2891874640 10 monclient(hunting): authenticate will time out at 854.201796s 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f288a7fc640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f288c0a49a0 0x7f288c13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:32998/0 (socket says 172.21.15.7:32998) 2024-05-08T15:47:26.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:26.087+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 learned_addr learned my addr 172.21.15.7:0/438846188 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:29.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient(hunting): tick 2024-05-08T15:47:29.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:47:29.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 1 monclient(hunting): continuing hunt 2024-05-08T15:47:29.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:47:29.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6791/0 conn(0x7f288c0a41b0 legacy=0x7f288c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:29.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6791/0 conn(0x7f288c0a49a0 legacy=0x7f288c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:29.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6789/0 conn(0x7f288c137940 legacy=0x7f288c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient: _add_conns ranks=[6,0,2,8,7,3] 2024-05-08T15:47:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f28780037e0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f2878007470 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f287800b1a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a39f0 con 0x7f2878007470 2024-05-08T15:47:29.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287800ffd0 con 0x7f287800b1a0 2024-05-08T15:47:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2878011260 con 0x7f28780037e0 2024-05-08T15:47:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f2888ff9640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6789/0 conn(0x7f2878007470 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:29.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.087+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f287800b1a0 legacy=0x7f287800d660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:29.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.288+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6789/0 conn(0x7f2878007470 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:29.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.288+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f287800b1a0 legacy=0x7f287800d660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:29.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.689+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6789/0 conn(0x7f2878007470 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:29.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:29.689+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f287800b1a0 legacy=0x7f287800d660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:30.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:30.490+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6789/0 conn(0x7f2878007470 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:30.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:30.490+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f287800b1a0 legacy=0x7f287800d660 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient(hunting): tick 2024-05-08T15:47:32.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 1 monclient(hunting): continuing hunt 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6792/0 conn(0x7f28780037e0 legacy=0x7f2878005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f287800b1a0 legacy=0x7f287800d660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6789/0 conn(0x7f2878007470 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient: _add_conns ranks=[8,3,7] 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 reap_dead start 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f2878011670 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 reap_dead start 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-g con 0x7f287800a0a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f287800a480 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28780124f0 con 0x7f287800a0a0 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a39f0 con 0x7f287800a480 2024-05-08T15:47:32.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287800ffd0 con 0x7f2878011670 2024-05-08T15:47:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f2888ff9640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:32.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.087+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:32.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.288+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:32.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.289+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:32.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.689+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:32.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:32.690+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:33.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:33.490+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:33.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:33.491+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient(hunting): tick 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 1 monclient(hunting): continuing hunt 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6790/0 conn(0x7f287800a0a0 legacy=0x7f287800d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient: _add_conns ranks=[8,7,2,1,0,4,6,5,3] 2024-05-08T15:47:35.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-e con 0x7f2878011670 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f287800a480 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-b con 0x7f2878005ef0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2878005c30 con 0x7f2878005ef0 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a39f0 con 0x7f287800a480 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f287800ffd0 con 0x7f2878011670 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f2888ff9640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f2878005ef0 legacy=0x7f287800a960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:35.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.088+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.289+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.289+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f2878005ef0 legacy=0x7f287800a960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:35.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.289+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.690+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.690+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f2878005ef0 legacy=0x7f287800a960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:35.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:35.690+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:36.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:36.491+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f2878005ef0 legacy=0x7f287800a960 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:36.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:36.491+0000 7f288affd640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:36.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:36.491+0000 7f288a7fc640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient(hunting): tick 2024-05-08T15:47:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:47:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 1 monclient(hunting): continuing hunt 2024-05-08T15:47:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:47:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6793/0 conn(0x7f2878011670 legacy=0x7f2878011ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6792/0 conn(0x7f287800a480 legacy=0x7f28780098d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:38.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6790/0 conn(0x7f2878005ef0 legacy=0x7f287800a960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient: _add_conns ranks=[4,1,6,3,0,5] 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f2878014970 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f2878018aa0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f287801c5b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2878017860 con 0x7f2878018aa0 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2878005c30 con 0x7f2878014970 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a39f0 con 0x7f287801c5b0 2024-05-08T15:47:38.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.088+0000 7f2888ff9640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692574824 0 0) 0x7f288c0a39f0 con 0x7f287801c5b0 2024-05-08T15:47:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient(hunting): my global_id is 74553 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2874003520 con 0x7f287801c5b0 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1730907457 0 0) 0x7f2874003520 con 0x7f287801c5b0 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6791/0 conn(0x7f2878014970 legacy=0x7f2878016d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.81:6789/0 conn(0x7f2878018aa0 legacy=0x7f287801ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 monclient: found mon.noname-d 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f288b7fe640 1 -- 172.21.15.7:0/438846188 reap_dead start 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287800ffd0 con 0x7f287801c5b0 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2884001b30 con 0x7f287801c5b0 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: dump: 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.206 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2891874640 5 monclient: authenticate success, global_id 74553 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2891874640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2884001d90 con 0x7f287801c5b0 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2891874640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2891874640 4 set_mon_vals no callback set 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 1 -- 172.21.15.7:0/438846188 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2884003cb0 con 0x7f287801c5b0 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.206+0000 7f2889ffb640 10 monclient: dump: 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.208 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.207+0000 7f2891874640 10 monclient: shutdown 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.207+0000 7f2891874640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.207+0000 7f2891874640 1 -- 172.21.15.7:0/438846188 >> v1:172.21.15.7:6792/0 conn(0x7f287801c5b0 legacy=0x7f287801e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.207+0000 7f2891874640 1 -- 172.21.15.7:0/438846188 shutdown_connections 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.207+0000 7f2891874640 1 -- 172.21.15.7:0/438846188 >> 172.21.15.7:0/438846188 conn(0x7f288c0a9fa0 msgr2=0x7f288c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.207+0000 7f2891874640 1 -- 172.21.15.7:0/438846188 shutdown_connections 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.207+0000 7f2891874640 1 -- 172.21.15.7:0/438846188 wait complete. 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.208+0000 7f2891874640 10 monclient: build_initial_monmap 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.208+0000 7f2891874640 10 monclient: monmap: 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:38.209 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:38.210 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.208+0000 7f2891874640 1 Processor -- start 2024-05-08T15:47:38.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.209+0000 7f2891874640 1 -- start start 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.209+0000 7f2891874640 10 monclient: init 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.209+0000 7f2891874640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.209+0000 7f2891874640 10 monclient: _add_conns ranks=[2,0,5,6,3,8,1,4,7] 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 10 monclient(hunting): picked mon.noname-b con 0x7f287801c5b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 10 monclient(hunting): picked mon.noname-a con 0x7f288c123c30 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 10 monclient(hunting): picked mon.noname-h con 0x7f288c1cd1b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f288affd640 1 -- >> v1:172.21.15.7:6790/0 conn(0x7f287801c5b0 legacy=0x7f288c1ccaa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f288a7fc640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7f288c123c30 legacy=0x7f288c1d1a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c1265d0 con 0x7f288c123c30 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c125400 con 0x7f287801c5b0 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f288c0a1f30 con 0x7f288c1cd1b0 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f2891874640 10 monclient(hunting): authenticate will time out at 866.324855s 2024-05-08T15:47:38.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f288b7fe640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7f288c1cd1b0 0x7f288c1d5170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:48922/0 (socket says 172.21.15.7:48922) 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.210+0000 7f288b7fe640 1 -- 172.21.15.7:0/1549943975 learned_addr learned my addr 172.21.15.7:0/1549943975 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.214+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087216618 0 0) 0x7f288c0a1f30 con 0x7f288c1cd1b0 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.214+0000 7f2873fff640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.214+0000 7f2873fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.214+0000 7f2873fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.214+0000 7f2873fff640 10 monclient(hunting): my global_id is 4577 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2878011430 con 0x7f288c1cd1b0 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4275873713 0 0) 0x7f2878011430 con 0x7f288c1cd1b0 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f288c0a1f30 con 0x7f288c1cd1b0 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2884001ab0 con 0x7f288c1cd1b0 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient(hunting): got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient(hunting): dump: 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.216 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 668667463 0 0) 0x7f288c0a1f30 con 0x7f288c1cd1b0 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 >> v1:172.21.15.7:6790/0 conn(0x7f287801c5b0 legacy=0x7f288c1ccaa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 >> v1:172.21.15.7:6789/0 conn(0x7f288c123c30 legacy=0x7f288c1d1a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 monclient: found mon.h 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288c1268b0 con 0x7f288c1cd1b0 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2891874640 5 monclient: authenticate success, global_id 4577 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2891874640 10 monclient: _renew_subs 2024-05-08T15:47:38.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2891874640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 --> v1:172.21.15.81:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f288c1cd590 con 0x7f288c1cd1b0 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2891874640 10 monclient: _renew_subs 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2891874640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2884003ea0 con 0x7f288c1cd1b0 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 --> v1:172.21.15.81:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f288c1d6890 con 0x7f288c1cd1b0 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2884004410 con 0x7f288c1cd1b0 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f2873fff640 10 monclient: dump: 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.219 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.215+0000 7f28897fa640 4 set_mon_vals no callback set 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.216+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f28840045b0 con 0x7f288c1cd1b0 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.216+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 8 ==== osd_map(19..19 src has 1..19) ==== 3898+0+0 (unknown 4243219874 0 0) 0x7f2884046f50 con 0x7f288c1cd1b0 2024-05-08T15:47:38.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.216+0000 7f2891874640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.217+0000 7f2891874640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.217+0000 7f2891874640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:38.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.217+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288c1248a0 con 0x7f288c1cd1b0 2024-05-08T15:47:38.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.217+0000 7f288affd640 10 monclient: get_auth_request con 0x7f287803c940 auth_method 0 2024-05-08T15:47:38.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.222+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f2884016020 con 0x7f288c1cd1b0 2024-05-08T15:47:38.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.222+0000 7f2873fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.222+0000 7f2873fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34025]: starting. 2024-05-08T15:47:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34025]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34024 2024-05-08T15:47:38.361 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34025]: created object 0... 2024-05-08T15:47:38.362 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34025]: created object 25... 2024-05-08T15:47:38.362 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34025]: created object 49... 2024-05-08T15:47:38.362 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34025]: finishing. 2024-05-08T15:47:38.362 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34025]: shutting down. 2024-05-08T15:47:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.508+0000 7f2891874640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:47:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.508+0000 7f2891874640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.508+0000 7f2891874640 10 monclient: _send_mon_message to mon.h at v1:172.21.15.81:6791/0 2024-05-08T15:47:38.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.508+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 --> v1:172.21.15.81:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f288c0753d0 con 0x7f288c1cd1b0 2024-05-08T15:47:38.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.509+0000 7f2873fff640 1 -- 172.21.15.7:0/1549943975 <== mon.5 v1:172.21.15.81:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2678 (unknown 254746657 0 3472584474) 0x7f2884015920 con 0x7f288c1cd1b0 2024-05-08T15:47:38.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.509+0000 7f2873fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:47:38.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.509+0000 7f2873fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:38.505 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":14,"quorum":[1,3,4,5,6],"quorum_names":["f","g","c","h","d"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:47:38.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 >> v1:172.21.15.81:6812/847627118 conn(0x7f287803c940 legacy=0x7f287803ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 10 monclient: shutdown 2024-05-08T15:47:38.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 >> v1:172.21.15.81:6791/0 conn(0x7f288c1cd1b0 legacy=0x7f288c1d5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 shutdown_connections 2024-05-08T15:47:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 >> 172.21.15.7:0/1549943975 conn(0x7f288c0a9fa0 msgr2=0x7f288c137040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 shutdown_connections 2024-05-08T15:47:38.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.511+0000 7f2891874640 1 -- 172.21.15.7:0/1549943975 wait complete. 2024-05-08T15:47:38.519 INFO:tasks.mon_thrash.ceph_manager:quorum is size 5 2024-05-08T15:47:38.519 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon c 2024-05-08T15:47:38.519 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:47:38.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.626+0000 7fa598811640 10 monclient: get_monmap_and_config 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.626+0000 7fa598811640 10 monclient: build_initial_monmap 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.626+0000 7fa598811640 10 monclient: monmap: 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:38.627062+0000 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:38.627062+0000 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.623 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.626+0000 7fa598811640 1 Processor -- start 2024-05-08T15:47:38.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.626+0000 7fa598811640 1 -- start start 2024-05-08T15:47:38.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.626+0000 7fa598811640 10 monclient: init 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 10 monclient: _add_conns ranks=[3,1,0,2,6,5,4,8,7] 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 10 monclient(hunting): picked mon.noname-g con 0x7fa5940a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 10 monclient(hunting): picked mon.noname-f con 0x7fa5940a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 10 monclient(hunting): picked mon.noname-a con 0x7fa59413e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5940a1660 con 0x7fa59413e310 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5940a28f0 con 0x7fa5940a5780 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5940a2bd0 con 0x7fa5940a53a0 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa598811640 10 monclient(hunting): authenticate will time out at 866.741991s 2024-05-08T15:47:38.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa592d9d640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7fa59413e310 legacy=0x7fa594140700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa59259c640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fa5940a53a0 0x7fa5940a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:38834/0 (socket says 172.21.15.7:38834) 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa591d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa5940a5780 0x7fa59413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:56568/0 (socket says 172.21.15.7:56568) 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa59259c640 1 -- 172.21.15.7:0/868360567 learned_addr learned my addr 172.21.15.7:0/868360567 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.627+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735693032 0 0) 0x7fa5940a28f0 con 0x7fa5940a5780 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): my global_id is 4492 2024-05-08T15:47:38.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa57c003520 con 0x7fa5940a5780 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676697104 0 0) 0x7fa5940a2bd0 con 0x7fa5940a53a0 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient(hunting): my global_id is 74559 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5940a28f0 con 0x7fa5940a53a0 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2321626070 0 0) 0x7fa57c003520 con 0x7fa5940a5780 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 >> v1:172.21.15.81:6790/0 conn(0x7fa5940a53a0 legacy=0x7fa5940a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 >> v1:172.21.15.7:6789/0 conn(0x7fa59413e310 legacy=0x7fa594140700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:38.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 monclient: found mon.noname-f 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5940a6730 con 0x7fa5940a5780 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa5880016e0 con 0x7fa5940a5780 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 5 monclient: authenticate success, global_id 4492 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: dump: 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.639 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa588001a00 con 0x7fa5940a5780 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 4 set_mon_vals no callback set 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 1 -- 172.21.15.7:0/868360567 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa588004b20 con 0x7fa5940a5780 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa59159a640 10 monclient: dump: 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.641 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 10 monclient: shutdown 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.628+0000 7fa598811640 1 -- 172.21.15.7:0/868360567 >> v1:172.21.15.81:6789/0 conn(0x7fa5940a5780 legacy=0x7fa59413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.629+0000 7fa598811640 1 -- 172.21.15.7:0/868360567 shutdown_connections 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.629+0000 7fa598811640 1 -- 172.21.15.7:0/868360567 >> 172.21.15.7:0/868360567 conn(0x7fa5940a9fa0 msgr2=0x7fa5940aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.629+0000 7fa598811640 1 -- 172.21.15.7:0/868360567 shutdown_connections 2024-05-08T15:47:38.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.629+0000 7fa598811640 1 -- 172.21.15.7:0/868360567 wait complete. 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.629+0000 7fa598811640 10 monclient: build_initial_monmap 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.629+0000 7fa598811640 10 monclient: monmap: 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:38.643 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 1 Processor -- start 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 1 -- start start 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient: init 2024-05-08T15:47:38.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient: _add_conns ranks=[7,5,6,4,2,3,1,8,0] 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient(hunting): picked mon.noname-i con 0x7fa5940a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient(hunting): picked mon.noname-h con 0x7fa5940a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient(hunting): picked mon.noname-d con 0x7fa59413e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5941ccf80 con 0x7fa5940a5780 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa594138730 con 0x7fa59413e310 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5940a1e00 con 0x7fa5940a53a0 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa598811640 10 monclient(hunting): authenticate will time out at 866.745313s 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa59259c640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7fa5940a53a0 legacy=0x7fa5941cc160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:38.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa592d9d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fa59413e310 0x7fa5941d46b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56076/0 (socket says 172.21.15.7:56076) 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa592d9d640 1 -- 172.21.15.7:0/3121246197 learned_addr learned my addr 172.21.15.7:0/3121246197 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615067755 0 0) 0x7fa594138730 con 0x7fa59413e310 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa582ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa582ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa582ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa582ffd640 10 monclient(hunting): my global_id is 74562 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.630+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa574003520 con 0x7fa59413e310 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1689261415 0 0) 0x7fa5941ccf80 con 0x7fa5940a5780 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): my global_id is 4586 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa594138730 con 0x7fa5940a5780 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2332109196 0 0) 0x7fa574003520 con 0x7fa59413e310 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa5941ccf80 con 0x7fa59413e310 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa58c004550 con 0x7fa59413e310 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.647 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1520571850 0 0) 0x7fa594138730 con 0x7fa5940a5780 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa574003520 con 0x7fa5940a5780 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa5880016a0 con 0x7fa5940a5780 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.649 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1834649175 0 0) 0x7fa5941ccf80 con 0x7fa59413e310 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.81:6792/0 conn(0x7fa5940a53a0 legacy=0x7fa5941cc160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.81:6791/0 conn(0x7fa5940a5780 legacy=0x7fa5941cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 monclient: found mon.d 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5940a43c0 con 0x7fa59413e310 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa598811640 5 monclient: authenticate success, global_id 74562 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa598811640 10 monclient: _renew_subs 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa598811640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa5941366c0 con 0x7fa59413e310 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa598811640 10 monclient: _renew_subs 2024-05-08T15:47:38.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa598811640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:47:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fa594133870 con 0x7fa59413e310 2024-05-08T15:47:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa58c0047b0 con 0x7fa59413e310 2024-05-08T15:47:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa58c004ca0 con 0x7fa59413e310 2024-05-08T15:47:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa590d99640 4 set_mon_vals no callback set 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.631+0000 7fa582ffd640 10 monclient: dump: 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.654 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa598811640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa5780065c0 con 0x7fa578002860 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa58c004f40 con 0x7fa59413e310 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(20..20 src has 1..20) ==== 3898+0+0 (unknown 1748080378 0 0) 0x7fa58c0470c0 con 0x7fa59413e310 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1505724328 0 0) 0x7fa5780065c0 con 0x7fa578002860 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 10 monclient(hunting): my global_id is 4459 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.632+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5941ccf80 con 0x7fa578002860 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa591d9b640 10 monclient: get_auth_request con 0x7fa57403a080 auth_method 0 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1228801372 0 0) 0x7fa5941ccf80 con 0x7fa578002860 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa5780065c0 con 0x7fa578002860 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa584001710 con 0x7fa578002860 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 10 monclient: dump: 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.656 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.657 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3893688508 0 0) 0x7fa5780065c0 con 0x7fa578002860 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa57800a150 con 0x7fa578002860 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fa584001cc0 con 0x7fa578002860 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.7:6791/0 conn(0x7fa578002860 legacy=0x7fa578004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.633+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.7:6791/0 conn(0x7fa578002860 legacy=0x7fa578004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa598811640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa578009e20 con 0x7fa5780090e0 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642445052 0 0) 0x7fa578009e20 con 0x7fa5780090e0 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 10 monclient(hunting): my global_id is 4468 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5780065c0 con 0x7fa5780090e0 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 50269725 0 0) 0x7fa5780065c0 con 0x7fa5780090e0 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa578009e20 con 0x7fa5780090e0 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa584004ea0 con 0x7fa5780090e0 2024-05-08T15:47:38.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 10 monclient: dump: 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.661 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2148585582 0 0) 0x7fa578009e20 con 0x7fa5780090e0 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.640+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa57800c390 con 0x7fa5780090e0 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7818 (unknown 0 0 1785417722) 0x7fa584007270 con 0x7fa5780090e0 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa582ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa582ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:38.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.7:6791/0 conn(0x7fa5780090e0 legacy=0x7fa578009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa582ffd640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.7:6791/0 conn(0x7fa5780090e0 legacy=0x7fa578009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 14, 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 50948, 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:38.663 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.665 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.666 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:38.667 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:38.668 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.669 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:38.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.671 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.672 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.673 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.675 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.676 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.677 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.678 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.679 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.681 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:38.682 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:38.683 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:38.684 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:47:38.685 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.686 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.687 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:38.687 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.687 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.687 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.687 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:38.687 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.81:6812/847627118 conn(0x7fa57403a080 legacy=0x7fa57403c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 10 monclient: shutdown 2024-05-08T15:47:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 >> v1:172.21.15.7:6792/0 conn(0x7fa59413e310 legacy=0x7fa5941d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa592d9d640 1 -- 172.21.15.7:0/3121246197 reap_dead start 2024-05-08T15:47:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa592d9d640 1 -- 172.21.15.7:0/3121246197 reap_dead start 2024-05-08T15:47:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 shutdown_connections 2024-05-08T15:47:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 >> 172.21.15.7:0/3121246197 conn(0x7fa5940a9fa0 msgr2=0x7fa5941389c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 shutdown_connections 2024-05-08T15:47:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.641+0000 7fa598811640 1 -- 172.21.15.7:0/3121246197 wait complete. 2024-05-08T15:47:38.689 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-05-08T15:47:38.689 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon d 2024-05-08T15:47:38.690 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:47:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.797+0000 7fa15104e640 10 monclient: get_monmap_and_config 2024-05-08T15:47:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.797+0000 7fa15104e640 10 monclient: build_initial_monmap 2024-05-08T15:47:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.797+0000 7fa15104e640 10 monclient: monmap: 2024-05-08T15:47:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:38.798149+0000 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:38.798149+0000 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:38.794 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.797+0000 7fa15104e640 1 Processor -- start 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.797+0000 7fa15104e640 1 -- start start 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.797+0000 7fa15104e640 10 monclient: init 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 10 monclient: _add_conns ranks=[2,3,7,6,0,8,4,5,1] 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 10 monclient(hunting): picked mon.noname-b con 0x7fa14c0a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 10 monclient(hunting): picked mon.noname-g con 0x7fa14c0a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 10 monclient(hunting): picked mon.noname-i con 0x7fa14c13e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c0a1660 con 0x7fa14c0a53a0 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c0a28f0 con 0x7fa14c0a5780 2024-05-08T15:47:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c0a2bd0 con 0x7fa14c13e310 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa15104e640 10 monclient(hunting): authenticate will time out at 866.913093s 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14ad9d640 1 -- >> v1:172.21.15.7:6790/0 conn(0x7fa14c0a53a0 legacy=0x7fa14c0a49a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14b59e640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7fa14c13e310 legacy=0x7fa14c140700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa143fff640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fa14c0a5780 0x7fa14c13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:38836/0 (socket says 172.21.15.7:38836) 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa143fff640 1 -- 172.21.15.7:0/2662912020 learned_addr learned my addr 172.21.15.7:0/2662912020 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430012841 0 0) 0x7fa14c0a28f0 con 0x7fa14c0a5780 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14a59c640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14a59c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14a59c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14a59c640 10 monclient(hunting): my global_id is 74568 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.798+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa138003520 con 0x7fa14c0a5780 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 283856374 0 0) 0x7fa138003520 con 0x7fa14c0a5780 2024-05-08T15:47:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 >> v1:172.21.15.81:6792/0 conn(0x7fa14c13e310 legacy=0x7fa14c140700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 >> v1:172.21.15.7:6790/0 conn(0x7fa14c0a53a0 legacy=0x7fa14c0a49a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 monclient: found mon.noname-g 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c0a6730 con 0x7fa14c0a5780 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa1340017b0 con 0x7fa14c0a5780 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: dump: 2024-05-08T15:47:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.805 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 5 monclient: authenticate success, global_id 74568 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa134001b10 con 0x7fa14c0a5780 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 4 set_mon_vals no callback set 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 1 -- 172.21.15.7:0/2662912020 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa134001d40 con 0x7fa14c0a5780 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa14a59c640 10 monclient: dump: 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 10 monclient: shutdown 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 1 -- 172.21.15.7:0/2662912020 >> v1:172.21.15.81:6790/0 conn(0x7fa14c0a5780 legacy=0x7fa14c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 1 -- 172.21.15.7:0/2662912020 shutdown_connections 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 1 -- 172.21.15.7:0/2662912020 >> 172.21.15.7:0/2662912020 conn(0x7fa14c0a9fa0 msgr2=0x7fa14c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 1 -- 172.21.15.7:0/2662912020 shutdown_connections 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.799+0000 7fa15104e640 1 -- 172.21.15.7:0/2662912020 wait complete. 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.800+0000 7fa15104e640 10 monclient: build_initial_monmap 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.800+0000 7fa15104e640 10 monclient: monmap: 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.800+0000 7fa15104e640 1 Processor -- start 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.800+0000 7fa15104e640 1 -- start start 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.800+0000 7fa15104e640 10 monclient: init 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 10 monclient: _add_conns ranks=[4,8,5,6,0,1,7,3,2] 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 10 monclient(hunting): picked mon.noname-c con 0x7fa14c0a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 10 monclient(hunting): picked mon.noname-e con 0x7fa14c0a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 10 monclient(hunting): picked mon.noname-h con 0x7fa14c13e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c0a4380 con 0x7fa14c0a53a0 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa143fff640 1 -- >> v1:172.21.15.7:6793/0 conn(0x7fa14c0a5780 legacy=0x7fa14c1cc950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c1cd060 con 0x7fa14c13e310 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa14c1368f0 con 0x7fa14c0a5780 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa15104e640 10 monclient(hunting): authenticate will time out at 866.916144s 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa14ad9d640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fa14c0a53a0 0x7fa14c1cc240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:53188/0 (socket says 172.21.15.7:53188) 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa14ad9d640 1 -- 172.21.15.7:0/3397986576 learned_addr learned my addr 172.21.15.7:0/3397986576 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 760829749 0 0) 0x7fa14c0a4380 con 0x7fa14c0a53a0 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): my global_id is 4477 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa12c003520 con 0x7fa14c0a53a0 2024-05-08T15:47:38.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145278193 0 0) 0x7fa14c1cd060 con 0x7fa14c13e310 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 10 monclient(hunting): my global_id is 4595 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa14c0a4380 con 0x7fa14c13e310 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.801+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1780628677 0 0) 0x7fa12c003520 con 0x7fa14c0a53a0 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa14c1cd060 con 0x7fa14c0a53a0 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa13c0042f0 con 0x7fa14c0a53a0 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient(hunting): dump: 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.811 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4038758956 0 0) 0x7fa14c1cd060 con 0x7fa14c0a53a0 2024-05-08T15:47:38.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.7:6793/0 conn(0x7fa14c0a5780 legacy=0x7fa14c1cc950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.81:6791/0 conn(0x7fa14c13e310 legacy=0x7fa14c1d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 monclient: found mon.c 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa14c135600 con 0x7fa14c0a53a0 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa15104e640 5 monclient: authenticate success, global_id 4477 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa15104e640 10 monclient: _renew_subs 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa15104e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa14c137050 con 0x7fa14c0a53a0 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa15104e640 10 monclient: _renew_subs 2024-05-08T15:47:38.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa15104e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fa14c136160 con 0x7fa14c0a53a0 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa13c0045a0 con 0x7fa14c0a53a0 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa13c004b10 con 0x7fa14c0a53a0 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa149d9b640 4 set_mon_vals no callback set 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.802+0000 7fa1437fe640 10 monclient: dump: 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.814 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa15104e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa1300065c0 con 0x7fa130002860 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa13c004cb0 con 0x7fa14c0a53a0 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(20..20 src has 1..20) ==== 3898+0+0 (unknown 1748080378 0 0) 0x7fa13c046df0 con 0x7fa14c0a53a0 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1131816477 0 0) 0x7fa1300065c0 con 0x7fa130002860 2024-05-08T15:47:38.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 10 monclient(hunting): my global_id is 74571 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa14c1cd060 con 0x7fa130002860 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa14b59e640 10 monclient: get_auth_request con 0x7fa12c039960 auth_method 0 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 680489379 0 0) 0x7fa14c1cd060 con 0x7fa130002860 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa1300065c0 con 0x7fa130002860 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa134001210 con 0x7fa130002860 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.803+0000 7fa1437fe640 10 monclient: dump: 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.816 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.804+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 144962610 0 0) 0x7fa1300065c0 con 0x7fa130002860 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.804+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa13000a150 con 0x7fa130002860 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.804+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fa134001590 con 0x7fa130002860 2024-05-08T15:47:38.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.804+0000 7fa1437fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.804+0000 7fa1437fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.804+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.7:6792/0 conn(0x7fa130002860 legacy=0x7fa130004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.804+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.7:6792/0 conn(0x7fa130002860 legacy=0x7fa130004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.810+0000 7fa15104e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.810+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa130009e20 con 0x7fa1300090e0 2024-05-08T15:47:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3526381034 0 0) 0x7fa130009e20 con 0x7fa1300090e0 2024-05-08T15:47:38.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient(hunting): my global_id is 74580 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1300065c0 con 0x7fa1300090e0 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 314961050 0 0) 0x7fa1300065c0 con 0x7fa1300090e0 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa130009e20 con 0x7fa1300090e0 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa134004b10 con 0x7fa1300090e0 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient: dump: 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.820 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2708965970 0 0) 0x7fa130009e20 con 0x7fa1300090e0 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa13000c390 con 0x7fa1300090e0 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7648 (unknown 0 0 3611755218) 0x7fa1340075d0 con 0x7fa1300090e0 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:38.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.7:6792/0 conn(0x7fa1300090e0 legacy=0x7fa130009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.811+0000 7fa1437fe640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.7:6792/0 conn(0x7fa1300090e0 legacy=0x7fa130009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 14, 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 51125, 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:38.822 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.823 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:38.825 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.827 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:38.828 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:38.830 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:38.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:38.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.836 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:38.837 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.838 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:38.839 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.81:6812/847627118 conn(0x7fa12c039960 legacy=0x7fa12c03be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 10 monclient: shutdown 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 >> v1:172.21.15.7:6791/0 conn(0x7fa14c0a53a0 legacy=0x7fa14c1cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa14b59e640 1 -- 172.21.15.7:0/3397986576 reap_dead start 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa14b59e640 1 -- 172.21.15.7:0/3397986576 reap_dead start 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 shutdown_connections 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 >> 172.21.15.7:0/3397986576 conn(0x7fa14c0a9fa0 msgr2=0x7fa14c140b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 shutdown_connections 2024-05-08T15:47:38.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.812+0000 7fa15104e640 1 -- 172.21.15.7:0/3397986576 wait complete. 2024-05-08T15:47:38.842 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-05-08T15:47:38.842 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon f 2024-05-08T15:47:38.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.949+0000 7fd6b963e640 10 monclient: get_monmap_and_config 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.949+0000 7fd6b963e640 10 monclient: build_initial_monmap 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.950+0000 7fd6b963e640 10 monclient: monmap: 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:38.950984+0000 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:38.950984+0000 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:38.955 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.950+0000 7fd6b963e640 1 Processor -- start 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.950+0000 7fd6b963e640 1 -- start start 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.950+0000 7fd6b963e640 10 monclient: init 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient: _add_conns ranks=[6,3,7,1,0,2,4,8,5] 2024-05-08T15:47:38.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient(hunting): picked mon.noname-d con 0x7fd6b4137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient(hunting): picked mon.noname-g con 0x7fd6b40a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient(hunting): picked mon.noname-i con 0x7fd6b40a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b40a2c70 con 0x7fd6b40a49a0 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b40a2f50 con 0x7fd6b4137940 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b40a1670 con 0x7fd6b40a41b0 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient(hunting): authenticate will time out at 867.065803s 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b37fe640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7fd6b40a41b0 legacy=0x7fd6b4140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b2ffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fd6b4137940 0x7fd6b4137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:56100/0 (socket says 172.21.15.7:56100) 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b2ffd640 1 -- 172.21.15.7:0/1856295377 learned_addr learned my addr 172.21.15.7:0/1856295377 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2296589221 0 0) 0x7fd6b40a2f50 con 0x7fd6b4137940 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): my global_id is 74589 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd69c003520 con 0x7fd6b4137940 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990444173 0 0) 0x7fd6b40a2c70 con 0x7fd6b40a49a0 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient(hunting): my global_id is 74577 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6b40a2f50 con 0x7fd6b40a49a0 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1070500253 0 0) 0x7fd69c003520 con 0x7fd6b4137940 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 >> v1:172.21.15.81:6792/0 conn(0x7fd6b40a41b0 legacy=0x7fd6b4140610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 >> v1:172.21.15.81:6790/0 conn(0x7fd6b40a49a0 legacy=0x7fd6b413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 monclient: found mon.noname-d 2024-05-08T15:47:38.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b40a39f0 con 0x7fd6b4137940 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6a4001b30 con 0x7fd6b4137940 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 5 monclient: authenticate success, global_id 74589 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: dump: 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.959 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.960 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd6a4001d90 con 0x7fd6b4137940 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b963e640 4 set_mon_vals no callback set 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 1 -- 172.21.15.7:0/1856295377 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6a4004f60 con 0x7fd6b4137940 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.951+0000 7fd6b1ffb640 10 monclient: dump: 2024-05-08T15:47:38.961 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.962 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 10 monclient: shutdown 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 1 -- 172.21.15.7:0/1856295377 >> v1:172.21.15.7:6792/0 conn(0x7fd6b4137940 legacy=0x7fd6b4137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 1 -- 172.21.15.7:0/1856295377 shutdown_connections 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 1 -- 172.21.15.7:0/1856295377 >> 172.21.15.7:0/1856295377 conn(0x7fd6b40a9fa0 msgr2=0x7fd6b40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 1 -- 172.21.15.7:0/1856295377 shutdown_connections 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 1 -- 172.21.15.7:0/1856295377 wait complete. 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 10 monclient: build_initial_monmap 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.952+0000 7fd6b963e640 10 monclient: monmap: 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:38.963 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 1 Processor -- start 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 1 -- start start 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient: init 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient: _add_conns ranks=[7,4,5,1,8,0,2,6,3] 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient(hunting): picked mon.noname-i con 0x7fd6b40a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient(hunting): picked mon.noname-c con 0x7fd6b40a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient(hunting): picked mon.noname-h con 0x7fd6b4137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b4135bb0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b4134de0 con 0x7fd6b4137940 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6b40a1f30 con 0x7fd6b40a41b0 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b963e640 10 monclient(hunting): authenticate will time out at 867.068500s 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.953+0000 7fd6b2ffd640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7fd6b40a41b0 legacy=0x7fd6b4135fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b27fc640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fd6b40a49a0 0x7fd6b41d0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:53190/0 (socket says 172.21.15.7:53190) 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b27fc640 1 -- 172.21.15.7:0/1350053699 learned_addr learned my addr 172.21.15.7:0/1350053699 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3886761019 0 0) 0x7fd6b4135bb0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): my global_id is 4486 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6a0003520 con 0x7fd6b40a49a0 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4046794992 0 0) 0x7fd6b4134de0 con 0x7fd6b4137940 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): my global_id is 4604 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6b4135bb0 con 0x7fd6b4137940 2024-05-08T15:47:38.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1184311612 0 0) 0x7fd6a0003520 con 0x7fd6b40a49a0 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6b4134de0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6a8001b50 con 0x7fd6b40a49a0 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): dump: 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.967 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1805367846 0 0) 0x7fd6b4135bb0 con 0x7fd6b4137940 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6a0003520 con 0x7fd6b4137940 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6ac0041d0 con 0x7fd6b4137940 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient(hunting): dump: 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.968 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1247962593 0 0) 0x7fd6b4134de0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.81:6792/0 conn(0x7fd6b40a41b0 legacy=0x7fd6b4135fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.81:6791/0 conn(0x7fd6b4137940 legacy=0x7fd6b41d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 monclient: found mon.c 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b40a0f60 con 0x7fd6b40a49a0 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b963e640 5 monclient: authenticate success, global_id 4486 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b963e640 10 monclient: _renew_subs 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b963e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:38.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b41338c0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b963e640 10 monclient: _renew_subs 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b963e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fd6b41366e0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd6a80038c0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6a8003db0 con 0x7fd6b40a49a0 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd6b17fa640 4 set_mon_vals no callback set 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.954+0000 7fd69b7fe640 10 monclient: dump: 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.971 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.955+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd6a8003f50 con 0x7fd6b40a49a0 2024-05-08T15:47:38.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.955+0000 7fd6b963e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.955+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd6940065c0 con 0x7fd694002860 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.955+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(20..20 src has 1..20) ==== 3898+0+0 (unknown 1748080378 0 0) 0x7fd6a8046f10 con 0x7fd6b40a49a0 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.955+0000 7fd6b37fe640 10 monclient: get_auth_request con 0x7fd6a003a030 auth_method 0 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.955+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 81824212 0 0) 0x7fd6940065c0 con 0x7fd694002860 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient(hunting): my global_id is 4501 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6b4134de0 con 0x7fd694002860 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2970087020 0 0) 0x7fd6b4134de0 con 0x7fd694002860 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6940065c0 con 0x7fd694002860 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6a4001590 con 0x7fd694002860 2024-05-08T15:47:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient: dump: 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.974 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2227943407 0 0) 0x7fd6940065c0 con 0x7fd694002860 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd69400a150 con 0x7fd694002860 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fd6a4001f80 con 0x7fd694002860 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.81:6789/0 conn(0x7fd694002860 legacy=0x7fd694004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.956+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.81:6789/0 conn(0x7fd694002860 legacy=0x7fd694004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd6b963e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd694009e20 con 0x7fd6940090e0 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1443372627 0 0) 0x7fd694009e20 con 0x7fd6940090e0 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd69b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd69b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd69b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd69b7fe640 10 monclient(hunting): my global_id is 4510 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.963+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6940065c0 con 0x7fd6940090e0 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2783651969 0 0) 0x7fd6940065c0 con 0x7fd6940090e0 2024-05-08T15:47:38.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd694009e20 con 0x7fd6940090e0 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6a4005510 con 0x7fd6940090e0 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 10 monclient: dump: 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:38.978 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 609917014 0 0) 0x7fd694009e20 con 0x7fd6940090e0 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd69400c390 con 0x7fd6940090e0 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7650 (unknown 0 0 259130814) 0x7fd6a4008680 con 0x7fd6940090e0 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.81:6789/0 conn(0x7fd6940090e0 legacy=0x7fd694009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:38.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.964+0000 7fd69b7fe640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.81:6789/0 conn(0x7fd6940090e0 legacy=0x7fd694009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 14, 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 51280, 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:38.980 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.981 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.983 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:38.985 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:38.986 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:38.987 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.988 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.992 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.993 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:38.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:38.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:38.998 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:38.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:39.000 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.81:6812/847627118 conn(0x7fd6a003a030 legacy=0x7fd6a003c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 10 monclient: shutdown 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 >> v1:172.21.15.7:6791/0 conn(0x7fd6b40a49a0 legacy=0x7fd6b41d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b37fe640 1 -- 172.21.15.7:0/1350053699 reap_dead start 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b37fe640 1 -- 172.21.15.7:0/1350053699 reap_dead start 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 shutdown_connections 2024-05-08T15:47:39.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 >> 172.21.15.7:0/1350053699 conn(0x7fd6b40a9fa0 msgr2=0x7fd6b41308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 shutdown_connections 2024-05-08T15:47:39.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:38.965+0000 7fd6b963e640 1 -- 172.21.15.7:0/1350053699 wait complete. 2024-05-08T15:47:39.003 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-05-08T15:47:39.003 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon g 2024-05-08T15:47:39.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:47:39.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.111+0000 7f2e88e2e640 10 monclient: get_monmap_and_config 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.111+0000 7f2e88e2e640 10 monclient: build_initial_monmap 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.111+0000 7f2e88e2e640 10 monclient: monmap: 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:39.112363+0000 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:39.112363+0000 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:39.108 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:39.109 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.111+0000 7f2e88e2e640 1 Processor -- start 2024-05-08T15:47:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 1 -- start start 2024-05-08T15:47:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient: init 2024-05-08T15:47:39.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient: _add_conns ranks=[2,7,3,1,5,6,8,0,4] 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient(hunting): picked mon.noname-b con 0x7f2e84137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient(hunting): picked mon.noname-i con 0x7f2e840a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient(hunting): picked mon.noname-g con 0x7f2e840a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e840a2c70 con 0x7f2e84137940 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e840a2f50 con 0x7f2e840a41b0 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e840a1670 con 0x7f2e840a49a0 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e88e2e640 10 monclient(hunting): authenticate will time out at 867.227311s 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e8259c640 1 -- >> v1:172.21.15.7:6790/0 conn(0x7f2e84137940 legacy=0x7f2e84137d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6790/0 2024-05-08T15:47:39.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.112+0000 7f2e81d9b640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7f2e840a49a0 legacy=0x7f2e8413cee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e82d9d640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f2e840a41b0 0x7f2e84140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:38858/0 (socket says 172.21.15.7:38858) 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e82d9d640 1 -- 172.21.15.7:0/3402717946 learned_addr learned my addr 172.21.15.7:0/3402717946 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3631581480 0 0) 0x7f2e840a2f50 con 0x7f2e840a41b0 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient(hunting): my global_id is 74586 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e6c003520 con 0x7f2e840a41b0 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3957997552 0 0) 0x7f2e6c003520 con 0x7f2e840a41b0 2024-05-08T15:47:39.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 >> v1:172.21.15.81:6792/0 conn(0x7f2e840a49a0 legacy=0x7f2e8413cee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 >> v1:172.21.15.7:6790/0 conn(0x7f2e84137940 legacy=0x7f2e84137d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: _finish_hunting 0 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 monclient: found mon.noname-g 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e840a39f0 con 0x7f2e840a41b0 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: _finish_auth 0 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: _check_auth_tickets 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:39.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e7c001b30 con 0x7f2e840a41b0 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e88e2e640 5 monclient: authenticate success, global_id 74586 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: dump: 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.124 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e88e2e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2e7c001e90 con 0x7f2e840a41b0 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e88e2e640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:39.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e88e2e640 4 set_mon_vals no callback set 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 1 -- 172.21.15.7:0/3402717946 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e7c0020c0 con 0x7f2e840a41b0 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.113+0000 7f2e8159a640 10 monclient: dump: 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.126 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 10 monclient: shutdown 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:39.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 1 -- 172.21.15.7:0/3402717946 >> v1:172.21.15.81:6790/0 conn(0x7f2e840a41b0 legacy=0x7f2e84140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 1 -- 172.21.15.7:0/3402717946 shutdown_connections 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 1 -- 172.21.15.7:0/3402717946 >> 172.21.15.7:0/3402717946 conn(0x7f2e840a9fa0 msgr2=0x7f2e840aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 1 -- 172.21.15.7:0/3402717946 shutdown_connections 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 1 -- 172.21.15.7:0/3402717946 wait complete. 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 10 monclient: build_initial_monmap 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.114+0000 7f2e88e2e640 10 monclient: monmap: 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:39.128 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 1 Processor -- start 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 1 -- start start 2024-05-08T15:47:39.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient: init 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient: _add_conns ranks=[1,5,3,7,0,6,4,2,8] 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient(hunting): picked mon.noname-f con 0x7f2e840a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient(hunting): picked mon.noname-h con 0x7f2e840a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient(hunting): picked mon.noname-g con 0x7f2e84137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e84136fb0 con 0x7f2e840a41b0 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e84139420 con 0x7f2e84137940 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e840a1f30 con 0x7f2e840a49a0 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.115+0000 7f2e88e2e640 10 monclient(hunting): authenticate will time out at 867.230418s 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e82d9d640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f2e84137940 0x7f2e841d4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:38874/0 (socket says 172.21.15.7:38874) 2024-05-08T15:47:39.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e82d9d640 1 -- 172.21.15.7:0/1271464611 learned_addr learned my addr 172.21.15.7:0/1271464611 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 749331760 0 0) 0x7f2e84139420 con 0x7f2e84137940 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): my global_id is 74595 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e64003520 con 0x7f2e84137940 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2695091836 0 0) 0x7f2e840a1f30 con 0x7f2e840a49a0 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): my global_id is 4613 2024-05-08T15:47:39.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e84139420 con 0x7f2e840a49a0 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 518957291 0 0) 0x7f2e84136fb0 con 0x7f2e840a41b0 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): my global_id is 4519 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e840a1f30 con 0x7f2e840a41b0 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 478872499 0 0) 0x7f2e64003520 con 0x7f2e84137940 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e84136fb0 con 0x7f2e84137940 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e7c001af0 con 0x7f2e84137940 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:47:39.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.133 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 404069616 0 0) 0x7f2e84139420 con 0x7f2e840a49a0 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e64003520 con 0x7f2e840a49a0 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2810558304 0 0) 0x7f2e840a1f30 con 0x7f2e840a41b0 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e84139420 con 0x7f2e840a41b0 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e78004250 con 0x7f2e840a49a0 2024-05-08T15:47:39.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.116+0000 7f2e72ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.135 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e74004160 con 0x7f2e840a41b0 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient(hunting): dump: 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.136 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.137 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1988627256 0 0) 0x7f2e84136fb0 con 0x7f2e84137940 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6791/0 conn(0x7f2e840a49a0 legacy=0x7f2e841d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6789/0 conn(0x7f2e840a41b0 legacy=0x7f2e84140c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 monclient: found mon.g 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e84139140 con 0x7f2e84137940 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: _finish_auth 0 2024-05-08T15:47:39.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 5 monclient: authenticate success, global_id 74595 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 10 monclient: _renew_subs 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2e84136c80 con 0x7f2e84137940 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 10 monclient: _renew_subs 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2e84141580 con 0x7f2e84137940 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2e7c0020a0 con 0x7f2e84137940 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e7c001e90 con 0x7f2e84137940 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e80d99640 4 set_mon_vals no callback set 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e72ffd640 10 monclient: dump: 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.140 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.117+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2e680065c0 con 0x7f2e68002860 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2e7c0108a0 con 0x7f2e84137940 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(20..20 src has 1..20) ==== 3898+0+0 (unknown 1748080378 0 0) 0x7f2e7c0472e0 con 0x7f2e84137940 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 332110504 0 0) 0x7f2e680065c0 con 0x7f2e68002860 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 10 monclient(hunting): my global_id is 74604 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e84136fb0 con 0x7f2e68002860 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e81d9b640 10 monclient: get_auth_request con 0x7f2e6403a4d0 auth_method 0 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3064664604 0 0) 0x7f2e84136fb0 con 0x7f2e68002860 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e680065c0 con 0x7f2e68002860 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e74004900 con 0x7f2e68002860 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 10 monclient: dump: 2024-05-08T15:47:39.142 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.143 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3603056334 0 0) 0x7f2e680065c0 con 0x7f2e68002860 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.118+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2e6800a150 con 0x7f2e68002860 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.119+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2e74004d10 con 0x7f2e68002860 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.119+0000 7f2e72ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.119+0000 7f2e72ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.119+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6790/0 conn(0x7f2e68002860 legacy=0x7f2e68004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.119+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6790/0 conn(0x7f2e68002860 legacy=0x7f2e68004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.125+0000 7f2e88e2e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.125+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2e68009e40 con 0x7f2e68009100 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131272721 0 0) 0x7f2e68009e40 con 0x7f2e68009100 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 10 monclient(hunting): my global_id is 74613 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e680065c0 con 0x7f2e68009100 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2629912973 0 0) 0x7f2e680065c0 con 0x7f2e68009100 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2e68009e40 con 0x7f2e68009100 2024-05-08T15:47:39.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2e74005570 con 0x7f2e68009100 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 10 monclient: dump: 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.147 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2596071030 0 0) 0x7f2e68009e40 con 0x7f2e68009100 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.126+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2e6800c3d0 con 0x7f2e68009100 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7478 (unknown 0 0 3523576606) 0x7f2e740069a0 con 0x7f2e68009100 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e72ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e72ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:39.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6790/0 conn(0x7f2e68009100 legacy=0x7f2e68009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e72ffd640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6790/0 conn(0x7f2e68009100 legacy=0x7f2e68009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 14, 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 51416, 2024-05-08T15:47:39.149 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:39.150 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.151 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:39.152 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:39.153 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:39.154 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.155 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:39.164 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:39.165 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:39.169 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.170 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:39.171 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6812/847627118 conn(0x7f2e6403a4d0 legacy=0x7f2e6403c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 10 monclient: shutdown 2024-05-08T15:47:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 >> v1:172.21.15.81:6790/0 conn(0x7f2e84137940 legacy=0x7f2e841d4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e82d9d640 1 -- 172.21.15.7:0/1271464611 reap_dead start 2024-05-08T15:47:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e82d9d640 1 -- 172.21.15.7:0/1271464611 reap_dead start 2024-05-08T15:47:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 shutdown_connections 2024-05-08T15:47:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 >> 172.21.15.7:0/1271464611 conn(0x7f2e840a9fa0 msgr2=0x7f2e840a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 shutdown_connections 2024-05-08T15:47:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.127+0000 7f2e88e2e640 1 -- 172.21.15.7:0/1271464611 wait complete. 2024-05-08T15:47:39.173 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-05-08T15:47:39.173 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 5 for mon h 2024-05-08T15:47:39.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:47:39.273 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34026]: starting. 2024-05-08T15:47:39.274 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34026]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34024 2024-05-08T15:47:39.274 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34026]: shutting down. 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.288+0000 7fec51779640 10 monclient: get_monmap_and_config 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.288+0000 7fec51779640 10 monclient: build_initial_monmap 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.288+0000 7fec51779640 10 monclient: monmap: 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:47:39.289694+0000 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:47:39.289694+0000 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:39.285 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:39.286 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:39.286 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec51779640 1 Processor -- start 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec51779640 1 -- start start 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec51779640 10 monclient: init 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec51779640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec51779640 10 monclient: _add_conns ranks=[8,0,4,2,1,3,6,7,5] 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec51779640 10 monclient(hunting): picked mon.noname-e con 0x7fec4c0a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec51779640 10 monclient(hunting): picked mon.noname-a con 0x7fec4c0a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 10 monclient(hunting): picked mon.noname-c con 0x7fec4c13e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.289+0000 7fec4affd640 1 -- >> v1:172.21.15.7:6793/0 conn(0x7fec4c0a53a0 legacy=0x7fec4c0a49a0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec4a7fc640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7fec4c0a5780 legacy=0x7fec4c13cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c0a1660 con 0x7fec4c0a5780 2024-05-08T15:47:39.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c0a28f0 con 0x7fec4c13e310 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c0a2bd0 con 0x7fec4c0a53a0 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 10 monclient(hunting): authenticate will time out at 867.404731s 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec4b7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fec4c13e310 0x7fec4c140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:53202/0 (socket says 172.21.15.7:53202) 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec4b7fe640 1 -- 172.21.15.7:0/4011151357 learned_addr learned my addr 172.21.15.7:0/4011151357 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 142210281 0 0) 0x7fec4c0a28f0 con 0x7fec4c13e310 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient(hunting): my global_id is 4495 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec34003520 con 0x7fec4c13e310 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 742925460 0 0) 0x7fec34003520 con 0x7fec4c13e310 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 >> v1:172.21.15.7:6793/0 conn(0x7fec4c0a53a0 legacy=0x7fec4c0a49a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 >> v1:172.21.15.7:6789/0 conn(0x7fec4c0a5780 legacy=0x7fec4c13cbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:47:39.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 monclient: found mon.noname-c 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c0a6730 con 0x7fec4c13e310 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: _finish_auth 0 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fec44001b30 con 0x7fec4c13e310 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: dump: 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.297 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 5 monclient: authenticate success, global_id 4495 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:47:39.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fec44001e90 con 0x7fec4c13e310 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 1 -- 172.21.15.7:0/4011151357 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fec440020c0 con 0x7fec4c13e310 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 10 monclient: get_monmap_and_config success 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec51779640 4 set_mon_vals no callback set 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.290+0000 7fec49ffb640 10 monclient: dump: 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.299 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 10 monclient: shutdown 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 1 -- 172.21.15.7:0/4011151357 >> v1:172.21.15.7:6791/0 conn(0x7fec4c13e310 legacy=0x7fec4c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 1 -- 172.21.15.7:0/4011151357 shutdown_connections 2024-05-08T15:47:39.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 1 -- 172.21.15.7:0/4011151357 >> 172.21.15.7:0/4011151357 conn(0x7fec4c0a9fa0 msgr2=0x7fec4c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 1 -- 172.21.15.7:0/4011151357 shutdown_connections 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 1 -- 172.21.15.7:0/4011151357 wait complete. 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 10 monclient: build_initial_monmap 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.291+0000 7fec51779640 10 monclient: monmap: 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:47:39.301 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 1 Processor -- start 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 1 -- start start 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 10 monclient: init 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 10 monclient: _reopen_session rank -1 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 10 monclient: _add_conns ranks=[0,8,4,3,5,7,1,2,6] 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 10 monclient(hunting): picked mon.noname-a con 0x7fec4c0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 10 monclient(hunting): picked mon.noname-e con 0x7fec4c0a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 10 monclient(hunting): picked mon.noname-c con 0x7fec4c13e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:47:39.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec4affd640 1 -- >> v1:172.21.15.7:6789/0 conn(0x7fec4c0a53a0 legacy=0x7fec4c1c9100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6789/0 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec4a7fc640 1 -- >> v1:172.21.15.7:6793/0 conn(0x7fec4c0a5780 legacy=0x7fec4c1d1560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c0a43c0 con 0x7fec4c0a53a0 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.292+0000 7fec51779640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c137010 con 0x7fec4c13e310 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec4c1362d0 con 0x7fec4c0a5780 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 10 monclient(hunting): _renew_subs 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 10 monclient(hunting): authenticate will time out at 867.407654s 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec4b7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fec4c13e310 0x7fec4c1d4c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:53214/0 (socket says 172.21.15.7:53214) 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec4b7fe640 1 -- 172.21.15.7:0/3079893511 learned_addr learned my addr 172.21.15.7:0/3079893511 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191938019 0 0) 0x7fec4c137010 con 0x7fec4c13e310 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): my global_id is 4504 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec38003520 con 0x7fec4c13e310 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1966302748 0 0) 0x7fec38003520 con 0x7fec4c13e310 2024-05-08T15:47:39.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fec4c137010 con 0x7fec4c13e310 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fec44001af0 con 0x7fec4c13e310 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): dump: 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.304 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3745857290 0 0) 0x7fec4c137010 con 0x7fec4c13e310 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.7:6793/0 conn(0x7fec4c0a5780 legacy=0x7fec4c1d1560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.7:6789/0 conn(0x7fec4c0a53a0 legacy=0x7fec4c1c9100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: _finish_hunting 0 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 monclient: found mon.c 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c138040 con 0x7fec4c13e310 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: _finish_auth 0 2024-05-08T15:47:39.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: _check_auth_tickets 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 5 monclient: authenticate success, global_id 4504 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 10 monclient: _renew_subs 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c135590 con 0x7fec4c13e310 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 10 monclient: _renew_subs 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fec4c1c99f0 con 0x7fec4c13e310 2024-05-08T15:47:39.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fec440052e0 con 0x7fec4c13e310 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fec44001e90 con 0x7fec4c13e310 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec497fa640 4 set_mon_vals no callback set 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.293+0000 7fec337fe640 10 monclient: dump: 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.307 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec51779640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fec2c0065c0 con 0x7fec2c002860 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fec440109b0 con 0x7fec4c13e310 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(21..21 src has 1..21) ==== 3506+0+0 (unknown 545282323 0 0) 0x7fec440471b0 con 0x7fec4c13e310 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530511931 0 0) 0x7fec2c0065c0 con 0x7fec2c002860 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 10 monclient(hunting): my global_id is 4640 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec4c137010 con 0x7fec2c002860 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.294+0000 7fec4a7fc640 10 monclient: get_auth_request con 0x7fec38039a90 auth_method 0 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3240919868 0 0) 0x7fec4c137010 con 0x7fec2c002860 2024-05-08T15:47:39.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fec2c0065c0 con 0x7fec2c002860 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fec3c004100 con 0x7fec2c002860 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 10 monclient: dump: 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.310 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 842217991 0 0) 0x7fec2c0065c0 con 0x7fec2c002860 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec2c00a150 con 0x7fec2c002860 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fec3c006510 con 0x7fec2c002860 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:47:39.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:47:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.81:6791/0 conn(0x7fec2c002860 legacy=0x7fec2c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.295+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.81:6791/0 conn(0x7fec2c002860 legacy=0x7fec2c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec51779640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:47:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fec2c009e40 con 0x7fec2c009100 2024-05-08T15:47:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2283720098 0 0) 0x7fec2c009e40 con 0x7fec2c009100 2024-05-08T15:47:39.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec337fe640 10 monclient(hunting): _negotiate 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec337fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec337fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec337fe640 10 monclient(hunting): my global_id is 4649 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.302+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec2c0065c0 con 0x7fec2c009100 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1560797466 0 0) 0x7fec2c0065c0 con 0x7fec2c009100 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fec2c009e40 con 0x7fec2c009100 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fec3c004ae0 con 0x7fec2c009100 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 10 monclient: dump: 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:47:39.314 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:47:39.315 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2632366873 0 0) 0x7fec2c009e40 con 0x7fec2c009100 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fec2c00c3d0 con 0x7fec2c009100 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7648 (unknown 0 0 3950616528) 0x7fec3c005fb0 con 0x7fec2c009100 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.81:6791/0 conn(0x7fec2c009100 legacy=0x7fec2c009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.303+0000 7fec337fe640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.81:6791/0 conn(0x7fec2c009100 legacy=0x7fec2c009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:39.316 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 14, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 51609, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: 6 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:47:39.317 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:39.318 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:39.319 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:47:39.320 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:47:39.321 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.322 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.324 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.326 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:47:39.327 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:47:39.328 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.329 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.331 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:47:39.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:47:39.331 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.331 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:47:39.332 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:47:39.332 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:47:39.335 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.336 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:47:39.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:47:39.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:47:39.338 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:47:39.338 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:47:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.81:6812/847627118 conn(0x7fec38039a90 legacy=0x7fec3803bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 10 monclient: shutdown 2024-05-08T15:47:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:47:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 >> v1:172.21.15.7:6791/0 conn(0x7fec4c13e310 legacy=0x7fec4c1d4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:47:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec4b7fe640 1 -- 172.21.15.7:0/3079893511 reap_dead start 2024-05-08T15:47:39.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec4b7fe640 1 -- 172.21.15.7:0/3079893511 reap_dead start 2024-05-08T15:47:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 shutdown_connections 2024-05-08T15:47:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 >> 172.21.15.7:0/3079893511 conn(0x7fec4c0a9fa0 msgr2=0x7fec4c140aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:47:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 shutdown_connections 2024-05-08T15:47:39.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:47:39.304+0000 7fec51779640 1 -- 172.21.15.7:0/3079893511 wait complete. 2024-05-08T15:47:39.340 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 5 2024-05-08T15:47:39.340 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-05-08T15:47:43.418 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:43.418 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34059]: starting. 2024-05-08T15:47:43.418 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34059]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34024 2024-05-08T15:47:43.418 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34059]: created object 0... 2024-05-08T15:47:43.418 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34059]: created object 25... 2024-05-08T15:47:43.418 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34059]: created object 49... 2024-05-08T15:47:43.419 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34059]: finishing. 2024-05-08T15:47:43.419 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34059]: shutting down. 2024-05-08T15:47:43.428 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:43.428 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34061]: starting. 2024-05-08T15:47:43.429 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34061]: listing objects. 2024-05-08T15:47:43.429 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34061]: listed object 0... 2024-05-08T15:47:43.429 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34061]: listed object 25... 2024-05-08T15:47:43.429 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34061]: saw 50 objects 2024-05-08T15:47:43.429 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34061]: shutting down. 2024-05-08T15:47:44.340 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:44.340 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34060]: starting. 2024-05-08T15:47:44.340 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34060]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34024 2024-05-08T15:47:44.340 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34060]: shutting down. 2024-05-08T15:47:44.342 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:44.342 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:44.342 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:47:44.347 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:47:46.442 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34130]: starting. 2024-05-08T15:47:46.443 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34130]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34129 2024-05-08T15:47:46.443 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34130]: created object 0... 2024-05-08T15:47:46.443 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34130]: created object 25... 2024-05-08T15:47:46.443 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34130]: created object 49... 2024-05-08T15:47:46.443 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34130]: finishing. 2024-05-08T15:47:46.443 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34130]: shutting down. 2024-05-08T15:47:47.359 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34131]: starting. 2024-05-08T15:47:47.359 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34131]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34129 2024-05-08T15:47:47.359 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34131]: shutting down. 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34166]: starting. 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34166]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34129 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34166]: created object 0... 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34166]: created object 25... 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34166]: created object 49... 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34166]: finishing. 2024-05-08T15:47:51.532 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34166]: shutting down. 2024-05-08T15:47:51.542 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:51.543 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34168]: starting. 2024-05-08T15:47:51.543 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34168]: listing objects. 2024-05-08T15:47:51.543 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34168]: listed object 0... 2024-05-08T15:47:51.543 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34168]: listed object 25... 2024-05-08T15:47:51.543 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34168]: saw 50 objects 2024-05-08T15:47:51.543 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34168]: shutting down. 2024-05-08T15:47:52.444 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:52.444 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34167]: starting. 2024-05-08T15:47:52.444 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34167]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34129 2024-05-08T15:47:52.445 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34167]: shutting down. 2024-05-08T15:47:52.446 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:52.446 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:52.446 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:47:52.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:47:54.538 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34237]: starting. 2024-05-08T15:47:54.539 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34237]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34236 2024-05-08T15:47:54.539 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34237]: created object 0... 2024-05-08T15:47:54.539 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34237]: created object 25... 2024-05-08T15:47:54.539 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34237]: created object 49... 2024-05-08T15:47:54.539 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34237]: finishing. 2024-05-08T15:47:54.539 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34237]: shutting down. 2024-05-08T15:47:56.471 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34238]: starting. 2024-05-08T15:47:56.471 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34238]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34236 2024-05-08T15:47:56.471 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34238]: shutting down. 2024-05-08T15:47:58.572 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:58.572 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34273]: starting. 2024-05-08T15:47:58.572 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34273]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34236 2024-05-08T15:47:58.572 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34273]: created object 0... 2024-05-08T15:47:58.572 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34273]: created object 25... 2024-05-08T15:47:58.572 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34273]: created object 49... 2024-05-08T15:47:58.573 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34273]: finishing. 2024-05-08T15:47:58.573 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34273]: shutting down. 2024-05-08T15:47:59.513 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:47:59.513 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34275]: starting. 2024-05-08T15:47:59.513 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34275]: listing objects. 2024-05-08T15:47:59.514 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34275]: listed object 0... 2024-05-08T15:47:59.514 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34275]: listed object 25... 2024-05-08T15:47:59.514 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34275]: saw 50 objects 2024-05-08T15:47:59.514 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34275]: shutting down. 2024-05-08T15:48:00.505 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:00.505 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34274]: starting. 2024-05-08T15:48:00.505 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34274]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34236 2024-05-08T15:48:00.505 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34274]: shutting down. 2024-05-08T15:48:00.506 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:00.506 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:00.506 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:00.509 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:02.598 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34344]: starting. 2024-05-08T15:48:02.599 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34344]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34343 2024-05-08T15:48:02.599 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34344]: created object 0... 2024-05-08T15:48:02.599 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34344]: created object 25... 2024-05-08T15:48:02.599 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34344]: created object 49... 2024-05-08T15:48:02.599 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34344]: finishing. 2024-05-08T15:48:02.599 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34344]: shutting down. 2024-05-08T15:48:04.531 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34345]: starting. 2024-05-08T15:48:04.531 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34345]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34343 2024-05-08T15:48:04.531 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34345]: shutting down. 2024-05-08T15:48:06.645 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:06.645 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34380]: starting. 2024-05-08T15:48:06.645 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34380]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34343 2024-05-08T15:48:06.646 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34380]: created object 0... 2024-05-08T15:48:06.646 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34380]: created object 25... 2024-05-08T15:48:06.646 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34380]: created object 49... 2024-05-08T15:48:06.646 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34380]: finishing. 2024-05-08T15:48:06.646 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34380]: shutting down. 2024-05-08T15:48:06.656 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:06.656 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34382]: starting. 2024-05-08T15:48:06.657 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34382]: listing objects. 2024-05-08T15:48:06.657 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34382]: listed object 0... 2024-05-08T15:48:06.657 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34382]: listed object 25... 2024-05-08T15:48:06.657 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34382]: saw 50 objects 2024-05-08T15:48:06.657 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34382]: shutting down. 2024-05-08T15:48:07.543 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:07.544 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34381]: starting. 2024-05-08T15:48:07.544 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34381]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34343 2024-05-08T15:48:07.544 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34381]: shutting down. 2024-05-08T15:48:07.546 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:07.546 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:07.546 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:07.549 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:08.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34451]: starting. 2024-05-08T15:48:08.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34451]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34450 2024-05-08T15:48:08.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34451]: created object 0... 2024-05-08T15:48:08.795 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34451]: created object 25... 2024-05-08T15:48:08.795 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34451]: created object 49... 2024-05-08T15:48:08.795 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34451]: finishing. 2024-05-08T15:48:08.795 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34451]: shutting down. 2024-05-08T15:48:09.712 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34452]: starting. 2024-05-08T15:48:09.712 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34452]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34450 2024-05-08T15:48:09.712 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34452]: shutting down. 2024-05-08T15:48:11.797 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:11.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34487]: starting. 2024-05-08T15:48:11.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34487]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34450 2024-05-08T15:48:11.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34487]: created object 0... 2024-05-08T15:48:11.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34487]: created object 25... 2024-05-08T15:48:11.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34487]: created object 49... 2024-05-08T15:48:11.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34487]: finishing. 2024-05-08T15:48:11.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34487]: shutting down. 2024-05-08T15:48:12.749 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:12.749 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34489]: starting. 2024-05-08T15:48:12.750 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34489]: listing objects. 2024-05-08T15:48:12.750 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34489]: listed object 0... 2024-05-08T15:48:12.750 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34489]: listed object 25... 2024-05-08T15:48:12.750 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34489]: saw 50 objects 2024-05-08T15:48:12.750 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34489]: shutting down. 2024-05-08T15:48:13.741 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:13.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34488]: starting. 2024-05-08T15:48:13.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34488]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34450 2024-05-08T15:48:13.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34488]: shutting down. 2024-05-08T15:48:13.743 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:13.743 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:13.743 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:13.746 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:15.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34558]: starting. 2024-05-08T15:48:15.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34558]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34557 2024-05-08T15:48:15.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34558]: created object 0... 2024-05-08T15:48:15.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34558]: created object 25... 2024-05-08T15:48:15.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34558]: created object 49... 2024-05-08T15:48:15.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34558]: finishing. 2024-05-08T15:48:15.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34558]: shutting down. 2024-05-08T15:48:16.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34559]: starting. 2024-05-08T15:48:16.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34559]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34557 2024-05-08T15:48:16.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34559]: shutting down. 2024-05-08T15:48:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34594]: starting. 2024-05-08T15:48:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34594]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34557 2024-05-08T15:48:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34594]: created object 0... 2024-05-08T15:48:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34594]: created object 25... 2024-05-08T15:48:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34594]: created object 49... 2024-05-08T15:48:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34594]: finishing. 2024-05-08T15:48:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34594]: shutting down. 2024-05-08T15:48:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34596]: starting. 2024-05-08T15:48:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34596]: listing objects. 2024-05-08T15:48:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34596]: listed object 0... 2024-05-08T15:48:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34596]: listed object 25... 2024-05-08T15:48:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34596]: saw 50 objects 2024-05-08T15:48:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34596]: shutting down. 2024-05-08T15:48:22.788 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:22.788 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34595]: starting. 2024-05-08T15:48:22.788 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34595]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34557 2024-05-08T15:48:22.789 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34595]: shutting down. 2024-05-08T15:48:22.790 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:22.790 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:22.790 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:22.793 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34665]: starting. 2024-05-08T15:48:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34665]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34664 2024-05-08T15:48:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34665]: created object 0... 2024-05-08T15:48:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34665]: created object 25... 2024-05-08T15:48:24.892 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34665]: created object 49... 2024-05-08T15:48:24.892 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34665]: finishing. 2024-05-08T15:48:24.892 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34665]: shutting down. 2024-05-08T15:48:26.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34666]: starting. 2024-05-08T15:48:26.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34666]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34664 2024-05-08T15:48:26.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34666]: shutting down. 2024-05-08T15:48:28.803 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:28.804 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34701]: starting. 2024-05-08T15:48:28.804 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34701]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34664 2024-05-08T15:48:28.804 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34701]: created object 0... 2024-05-08T15:48:28.804 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34701]: created object 25... 2024-05-08T15:48:28.804 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34701]: created object 49... 2024-05-08T15:48:28.804 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34701]: finishing. 2024-05-08T15:48:28.804 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34701]: shutting down. 2024-05-08T15:48:28.814 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:28.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34703]: starting. 2024-05-08T15:48:28.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34703]: listing objects. 2024-05-08T15:48:28.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34703]: listed object 0... 2024-05-08T15:48:28.815 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34703]: listed object 25... 2024-05-08T15:48:28.815 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34703]: saw 50 objects 2024-05-08T15:48:28.815 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34703]: shutting down. 2024-05-08T15:48:29.707 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:29.707 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34702]: starting. 2024-05-08T15:48:29.707 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34702]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34664 2024-05-08T15:48:29.707 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34702]: shutting down. 2024-05-08T15:48:29.708 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:29.708 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:29.708 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:29.711 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:31.807 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34772]: starting. 2024-05-08T15:48:31.807 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34772]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34771 2024-05-08T15:48:31.807 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34772]: created object 0... 2024-05-08T15:48:31.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34772]: created object 25... 2024-05-08T15:48:31.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34772]: created object 49... 2024-05-08T15:48:31.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34772]: finishing. 2024-05-08T15:48:31.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34772]: shutting down. 2024-05-08T15:48:32.721 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34773]: starting. 2024-05-08T15:48:32.721 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34773]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34771 2024-05-08T15:48:32.721 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34773]: shutting down. 2024-05-08T15:48:37.839 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:37.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34808]: starting. 2024-05-08T15:48:37.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34808]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34771 2024-05-08T15:48:37.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34808]: created object 0... 2024-05-08T15:48:37.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34808]: created object 25... 2024-05-08T15:48:37.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34808]: created object 49... 2024-05-08T15:48:37.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34808]: finishing. 2024-05-08T15:48:37.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34808]: shutting down. 2024-05-08T15:48:37.850 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:37.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34810]: starting. 2024-05-08T15:48:37.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34810]: listing objects. 2024-05-08T15:48:37.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34810]: listed object 0... 2024-05-08T15:48:37.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34810]: listed object 25... 2024-05-08T15:48:37.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34810]: saw 50 objects 2024-05-08T15:48:37.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34810]: shutting down. 2024-05-08T15:48:38.757 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:38.757 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34809]: starting. 2024-05-08T15:48:38.758 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34809]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34771 2024-05-08T15:48:38.758 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34809]: shutting down. 2024-05-08T15:48:38.759 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:38.759 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:38.759 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:38.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:40.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34879]: starting. 2024-05-08T15:48:40.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34879]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34878 2024-05-08T15:48:40.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34879]: created object 0... 2024-05-08T15:48:40.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34879]: created object 25... 2024-05-08T15:48:40.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34879]: created object 49... 2024-05-08T15:48:40.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34879]: finishing. 2024-05-08T15:48:40.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34879]: shutting down. 2024-05-08T15:48:41.779 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34880]: starting. 2024-05-08T15:48:41.779 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34880]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34878 2024-05-08T15:48:41.780 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34880]: shutting down. 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34915]: starting. 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34915]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34878 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34915]: created object 0... 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34915]: created object 25... 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34915]: created object 49... 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34915]: finishing. 2024-05-08T15:48:43.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[34915]: shutting down. 2024-05-08T15:48:43.810 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:43.811 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34917]: starting. 2024-05-08T15:48:43.811 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34917]: listing objects. 2024-05-08T15:48:43.811 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34917]: listed object 0... 2024-05-08T15:48:43.811 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34917]: listed object 25... 2024-05-08T15:48:43.811 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34917]: saw 50 objects 2024-05-08T15:48:43.811 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[34917]: shutting down. 2024-05-08T15:48:44.709 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:44.709 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34916]: starting. 2024-05-08T15:48:44.709 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34916]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34878 2024-05-08T15:48:44.709 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[34916]: shutting down. 2024-05-08T15:48:44.710 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:44.711 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:44.711 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:44.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:49.824 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34986]: starting. 2024-05-08T15:48:49.824 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34986]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34985 2024-05-08T15:48:49.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34986]: created object 0... 2024-05-08T15:48:49.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34986]: created object 25... 2024-05-08T15:48:49.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34986]: created object 49... 2024-05-08T15:48:49.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34986]: finishing. 2024-05-08T15:48:49.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[34986]: shutting down. 2024-05-08T15:48:50.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34987]: starting. 2024-05-08T15:48:50.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34987]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34985 2024-05-08T15:48:50.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[34987]: shutting down. 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35022]: starting. 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35022]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-34985 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35022]: created object 0... 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35022]: created object 25... 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35022]: created object 49... 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35022]: finishing. 2024-05-08T15:48:55.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35022]: shutting down. 2024-05-08T15:48:55.859 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:55.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35024]: starting. 2024-05-08T15:48:55.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35024]: listing objects. 2024-05-08T15:48:55.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35024]: listed object 0... 2024-05-08T15:48:55.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35024]: listed object 25... 2024-05-08T15:48:55.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35024]: saw 50 objects 2024-05-08T15:48:55.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35024]: shutting down. 2024-05-08T15:48:56.765 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:56.766 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35023]: starting. 2024-05-08T15:48:56.766 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35023]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-34985 2024-05-08T15:48:56.766 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35023]: shutting down. 2024-05-08T15:48:56.767 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:56.767 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:48:56.767 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:48:56.770 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:48:58.801 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35095]: starting. 2024-05-08T15:48:58.801 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35095]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35094 2024-05-08T15:48:58.801 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35095]: created object 0... 2024-05-08T15:48:58.801 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35095]: created object 25... 2024-05-08T15:48:58.801 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35095]: created object 49... 2024-05-08T15:48:58.802 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35095]: finishing. 2024-05-08T15:48:58.802 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35095]: shutting down. 2024-05-08T15:48:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35096]: starting. 2024-05-08T15:48:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35096]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35094 2024-05-08T15:48:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35096]: shutting down. 2024-05-08T15:49:01.807 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:01.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35131]: starting. 2024-05-08T15:49:01.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35131]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35094 2024-05-08T15:49:01.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35131]: created object 0... 2024-05-08T15:49:01.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35131]: created object 25... 2024-05-08T15:49:01.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35131]: created object 49... 2024-05-08T15:49:01.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35131]: finishing. 2024-05-08T15:49:01.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35131]: shutting down. 2024-05-08T15:49:01.819 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:01.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35133]: starting. 2024-05-08T15:49:01.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35133]: listing objects. 2024-05-08T15:49:01.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35133]: listed object 0... 2024-05-08T15:49:01.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35133]: listed object 25... 2024-05-08T15:49:01.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35133]: saw 50 objects 2024-05-08T15:49:01.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35133]: shutting down. 2024-05-08T15:49:02.734 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:02.734 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35132]: starting. 2024-05-08T15:49:02.734 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35132]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35094 2024-05-08T15:49:02.734 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35132]: shutting down. 2024-05-08T15:49:02.736 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:02.736 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:02.736 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:49:02.738 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:49:04.830 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35202]: starting. 2024-05-08T15:49:04.830 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35202]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35201 2024-05-08T15:49:04.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35202]: created object 0... 2024-05-08T15:49:04.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35202]: created object 25... 2024-05-08T15:49:04.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35202]: created object 49... 2024-05-08T15:49:04.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35202]: finishing. 2024-05-08T15:49:04.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35202]: shutting down. 2024-05-08T15:49:05.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35203]: starting. 2024-05-08T15:49:05.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35203]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35201 2024-05-08T15:49:05.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35203]: shutting down. 2024-05-08T15:49:07.805 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:07.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35238]: starting. 2024-05-08T15:49:07.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35238]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35201 2024-05-08T15:49:07.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35238]: created object 0... 2024-05-08T15:49:07.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35238]: created object 25... 2024-05-08T15:49:07.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35238]: created object 49... 2024-05-08T15:49:07.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35238]: finishing. 2024-05-08T15:49:07.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35238]: shutting down. 2024-05-08T15:49:07.816 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:07.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35240]: starting. 2024-05-08T15:49:07.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35240]: listing objects. 2024-05-08T15:49:07.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35240]: listed object 0... 2024-05-08T15:49:07.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35240]: listed object 25... 2024-05-08T15:49:07.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35240]: saw 50 objects 2024-05-08T15:49:07.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35240]: shutting down. 2024-05-08T15:49:08.713 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:08.713 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35239]: starting. 2024-05-08T15:49:08.713 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35239]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35201 2024-05-08T15:49:08.713 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35239]: shutting down. 2024-05-08T15:49:08.714 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:08.714 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:08.714 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:49:08.716 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:49:09.341 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-05-08T15:49:09.342 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-05-08T15:49:09.342 INFO:tasks.ceph.mon.i:Restarting daemon 2024-05-08T15:49:09.342 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-05-08T15:49:09.345 INFO:tasks.ceph.mon.i:Started 2024-05-08T15:49:09.345 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-05-08T15:49:09.345 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-05-08T15:49:09.345 INFO:tasks.ceph.mon.e:Restarting daemon 2024-05-08T15:49:09.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-05-08T15:49:09.348 INFO:tasks.ceph.mon.e:Started 2024-05-08T15:49:09.348 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-05-08T15:49:09.348 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-05-08T15:49:09.348 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T15:49:09.348 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T15:49:09.351 INFO:tasks.ceph.mon.b:Started 2024-05-08T15:49:09.351 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-05-08T15:49:09.351 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-05-08T15:49:09.351 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T15:49:09.351 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T15:49:09.353 INFO:tasks.ceph.mon.a:Started 2024-05-08T15:49:09.353 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:49:09.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:49:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.485+0000 7f9bfeec2640 10 monclient: get_monmap_and_config 2024-05-08T15:49:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.485+0000 7f9bfeec2640 10 monclient: build_initial_monmap 2024-05-08T15:49:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.485+0000 7f9bfeec2640 10 monclient: monmap: 2024-05-08T15:49:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:09.481 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:09.486244+0000 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:09.486244+0000 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:09.482 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:09.483 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 1 Processor -- start 2024-05-08T15:49:09.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 1 -- start start 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 10 monclient: init 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 10 monclient: _add_conns ranks=[2,6,0,4,1,8,5,3,7] 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 10 monclient(hunting): picked mon.noname-b con 0x7f9bf80a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 10 monclient(hunting): picked mon.noname-d con 0x7f9bf80a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 10 monclient(hunting): picked mon.noname-a con 0x7f9bf813e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf80a1660 con 0x7f9bf813e310 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfeec2640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf80a28f0 con 0x7f9bf80a53a0 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfd45f640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f9bf813e310 0x7f9bf8140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:58220/0 (socket says 172.21.15.7:58220) 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.486+0000 7f9bfd45f640 1 -- 172.21.15.7:0/2232971838 learned_addr learned my addr 172.21.15.7:0/2232971838 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.487+0000 7f9bfeec2640 1 -- 172.21.15.7:0/2232971838 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf80a2bd0 con 0x7f9bf80a5780 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.487+0000 7f9bfeec2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:09.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:09.487+0000 7f9bfeec2640 10 monclient(hunting): authenticate will time out at 957.601665s 2024-05-08T15:49:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.486+0000 7f9bf67fc640 10 monclient(hunting): tick 2024-05-08T15:49:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.486+0000 7f9bf67fc640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:49:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.486+0000 7f9bf67fc640 1 monclient(hunting): continuing hunt 2024-05-08T15:49:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.486+0000 7f9bf67fc640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.486+0000 7f9bf67fc640 1 -- 172.21.15.7:0/2232971838 >> v1:172.21.15.7:6792/0 conn(0x7f9bf80a5780 legacy=0x7f9bf813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.486+0000 7f9bf67fc640 1 -- 172.21.15.7:0/2232971838 >> v1:172.21.15.7:6790/0 conn(0x7f9bf80a53a0 legacy=0x7f9bf80a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.486+0000 7f9bf67fc640 1 -- 172.21.15.7:0/2232971838 >> v1:172.21.15.7:6789/0 conn(0x7f9bf813e310 legacy=0x7f9bf8140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 10 monclient: _add_conns ranks=[5,7,4,8,1,3] 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 10 monclient(hunting): picked mon.noname-h con 0x7f9bd80037e0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 10 monclient(hunting): picked mon.noname-i con 0x7f9bd8007470 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 10 monclient(hunting): picked mon.noname-c con 0x7f9bd800b1c0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 1 -- 172.21.15.7:0/2232971838 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf80a6730 con 0x7f9bd800b1c0 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 1 -- 172.21.15.7:0/2232971838 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd800ffd0 con 0x7f9bd80037e0 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 1 -- 172.21.15.7:0/2232971838 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bd8011260 con 0x7f9bd8007470 2024-05-08T15:49:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:12.487+0000 7f9bf67fc640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.462+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3008086986 0 0) 0x7f9bd800ffd0 con 0x7f9bd80037e0 2024-05-08T15:49:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): my global_id is 4937 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be4003520 con 0x7f9bd80037e0 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 751693903 0 0) 0x7f9bf80a6730 con 0x7f9bd800b1c0 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient(hunting): my global_id is 4927 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bd800ffd0 con 0x7f9bd800b1c0 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1808156818 0 0) 0x7f9be4003520 con 0x7f9bd80037e0 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 >> v1:172.21.15.81:6792/0 conn(0x7f9bd8007470 legacy=0x7f9bd8009910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 >> v1:172.21.15.7:6791/0 conn(0x7f9bd800b1c0 legacy=0x7f9bd800d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 monclient: found mon.noname-h 2024-05-08T15:49:14.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bfd45f640 1 -- 172.21.15.7:0/2232971838 reap_dead start 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd80124f0 con 0x7f9bd80037e0 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9be00017f0 con 0x7f9bd80037e0 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bfeec2640 5 monclient: authenticate success, global_id 4937 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bfeec2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:49:14.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: dump: 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.465 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bfeec2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9be0001a50 con 0x7f9bd80037e0 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 1 -- 172.21.15.7:0/2232971838 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9be0003950 con 0x7f9bd80037e0 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bf77fe640 10 monclient: dump: 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.466 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bfeec2640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.463+0000 7f9bfeec2640 4 set_mon_vals no callback set 2024-05-08T15:49:14.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.464+0000 7f9bfeec2640 10 monclient: shutdown 2024-05-08T15:49:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.464+0000 7f9bfeec2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.464+0000 7f9bfeec2640 1 -- 172.21.15.7:0/2232971838 >> v1:172.21.15.81:6791/0 conn(0x7f9bd80037e0 legacy=0x7f9bd8005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.464+0000 7f9bfeec2640 1 -- 172.21.15.7:0/2232971838 shutdown_connections 2024-05-08T15:49:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.464+0000 7f9bfeec2640 1 -- 172.21.15.7:0/2232971838 >> 172.21.15.7:0/2232971838 conn(0x7f9bf80a9fa0 msgr2=0x7f9bf80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.464+0000 7f9bfeec2640 1 -- 172.21.15.7:0/2232971838 shutdown_connections 2024-05-08T15:49:14.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.464+0000 7f9bfeec2640 1 -- 172.21.15.7:0/2232971838 wait complete. 2024-05-08T15:49:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.465+0000 7f9bfeec2640 10 monclient: build_initial_monmap 2024-05-08T15:49:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.465+0000 7f9bfeec2640 10 monclient: monmap: 2024-05-08T15:49:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:14.469 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:14.470 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.466+0000 7f9bfeec2640 1 Processor -- start 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.466+0000 7f9bfeec2640 1 -- start start 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.466+0000 7f9bfeec2640 10 monclient: init 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 10 monclient: _add_conns ranks=[3,7,1,6,2,8,4,5,0] 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 10 monclient(hunting): picked mon.noname-g con 0x7f9bd80037e0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 10 monclient(hunting): picked mon.noname-i con 0x7f9bf809f010 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 10 monclient(hunting): picked mon.noname-f con 0x7f9bf81d2df0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf8122af0 con 0x7f9bf81d2df0 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf8120ae0 con 0x7f9bd80037e0 2024-05-08T15:49:14.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf81cd270 con 0x7f9bf809f010 2024-05-08T15:49:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfeec2640 10 monclient(hunting): authenticate will time out at 962.583050s 2024-05-08T15:49:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfcc5e640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7f9bd80037e0 0x7f9bf81ccb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:41550/0 (socket says 172.21.15.7:41550) 2024-05-08T15:49:14.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.467+0000 7f9bfcc5e640 1 -- 172.21.15.7:0/1798752868 learned_addr learned my addr 172.21.15.7:0/1798752868 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 710062845 0 0) 0x7f9bf8120ae0 con 0x7f9bd80037e0 2024-05-08T15:49:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): my global_id is 75018 2024-05-08T15:49:14.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bd8005330 con 0x7f9bd80037e0 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3185120054 0 0) 0x7f9bf81cd270 con 0x7f9bf809f010 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): my global_id is 94111 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bf8120ae0 con 0x7f9bf809f010 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1646219007 0 0) 0x7f9bd8005330 con 0x7f9bd80037e0 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9bf81cd270 con 0x7f9bd80037e0 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9be0003950 con 0x7f9bd80037e0 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf7fff640 1 -- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6792/0 conn(0x7f9bf809f010 legacy=0x7f9bf81d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf7fff640 1 -- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6792/0 conn(0x7f9bf809f010 legacy=0x7f9bf81d16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T15:49:14.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf7fff640 1 --1- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6792/0 conn(0x7f9bf809f010 0x7f9bf81d16d0 :-1 s=OPENED pgs=46 cs=1 l=1).handle_message read tag failed 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf7fff640 1 --1- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6792/0 conn(0x7f9bf809f010 0x7f9bf81d16d0 :-1 s=OPENED pgs=46 cs=1 l=1).fault on lossy channel, failing 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): dump: 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.477 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.81:6792/0 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1416215952 0 0) 0x7f9bf81cd270 con 0x7f9bd80037e0 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6792/0 conn(0x7f9bf809f010 legacy=0x7f9bf81d16d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6789/0 conn(0x7f9bf81d2df0 legacy=0x7f9bf81d51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient: _finish_hunting 0 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 monclient: found mon.g 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf8122810 con 0x7f9bd80037e0 2024-05-08T15:49:14.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient: _finish_auth 0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient: _check_auth_tickets 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bfeec2640 5 monclient: authenticate success, global_id 75018 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bfeec2640 10 monclient: _renew_subs 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bfeec2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf8122dd0 con 0x7f9bd80037e0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bfeec2640 10 monclient: _renew_subs 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bfeec2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9bf8137a90 con 0x7f9bd80037e0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9be0001d00 con 0x7f9bd80037e0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9be00041a0 con 0x7f9bd80037e0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.473+0000 7f9bf67fc640 4 set_mon_vals no callback set 2024-05-08T15:49:14.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.474+0000 7f9bf57fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.474+0000 7f9bf57fa640 10 monclient: dump: 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.480 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.474+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f9be0010980 con 0x7f9bd80037e0 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.474+0000 7f9bfeec2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.474+0000 7f9bfeec2640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.474+0000 7f9bfeec2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.474+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bdc005080 con 0x7f9bd80037e0 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.475+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(96..96 src has 1..96) ==== 3506+0+0 (unknown 3001059884 0 0) 0x7f9be0046d30 con 0x7f9bd80037e0 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.475+0000 7f9bf7fff640 10 monclient: get_auth_request con 0x7f9bd803b2e0 auth_method 0 2024-05-08T15:49:14.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.481+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f9be0015f20 con 0x7f9bd80037e0 2024-05-08T15:49:14.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.481+0000 7f9bf57fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:14.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.481+0000 7f9bf57fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:14.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.753+0000 7f9bfeec2640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:49:14.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.753+0000 7f9bfeec2640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:14.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.753+0000 7f9bfeec2640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:14.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.753+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9bdc005370 con 0x7f9bd80037e0 2024-05-08T15:49:14.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.753+0000 7f9bf57fa640 1 -- 172.21.15.7:0/1798752868 <== mon.3 v1:172.21.15.81:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 736146387) 0x7f9be0015820 con 0x7f9bd80037e0 2024-05-08T15:49:14.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.753+0000 7f9bf57fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:14.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.753+0000 7f9bf57fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:14.752 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":20,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6812/847627118 conn(0x7f9bd803b2e0 legacy=0x7f9bd803d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 10 monclient: shutdown 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 >> v1:172.21.15.81:6790/0 conn(0x7f9bd80037e0 legacy=0x7f9bf81ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 shutdown_connections 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 >> 172.21.15.7:0/1798752868 conn(0x7f9bf80a9fa0 msgr2=0x7f9bf80a9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 shutdown_connections 2024-05-08T15:49:14.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.755+0000 7f9bfeec2640 1 -- 172.21.15.7:0/1798752868 wait complete. 2024-05-08T15:49:14.765 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:49:14.765 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:49:14.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.872+0000 7fe81a2c8640 10 monclient: get_monmap_and_config 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.872+0000 7fe81a2c8640 10 monclient: build_initial_monmap 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.873+0000 7fe81a2c8640 10 monclient: monmap: 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:14.874972+0000 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:14.874972+0000 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:14.871 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.873+0000 7fe81a2c8640 1 Processor -- start 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.873+0000 7fe81a2c8640 1 -- start start 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.873+0000 7fe81a2c8640 10 monclient: init 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 10 monclient: _add_conns ranks=[4,1,3,8,6,7,0,5,2] 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 10 monclient(hunting): picked mon.noname-c con 0x7fe814137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 10 monclient(hunting): picked mon.noname-f con 0x7fe8140a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 10 monclient(hunting): picked mon.noname-g con 0x7fe8140a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8140a2c70 con 0x7fe8140a49a0 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8140a2f50 con 0x7fe8140a41b0 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8140a1670 con 0x7fe814137940 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe81a2c8640 10 monclient(hunting): authenticate will time out at 962.990063s 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe813fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fe814137940 0x7fe814137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57794/0 (socket says 172.21.15.7:57794) 2024-05-08T15:49:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe813fff640 1 -- 172.21.15.7:0/2107046542 learned_addr learned my addr 172.21.15.7:0/2107046542 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3593466447 0 0) 0x7fe8140a1670 con 0x7fe814137940 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): my global_id is 4936 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7fc003520 con 0x7fe814137940 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586961176 0 0) 0x7fe8140a2c70 con 0x7fe8140a49a0 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 10 monclient(hunting): my global_id is 4897 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.874+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8140a1670 con 0x7fe8140a49a0 2024-05-08T15:49:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1908570168 0 0) 0x7fe8140a2f50 con 0x7fe8140a41b0 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient(hunting): my global_id is 75027 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8140a2c70 con 0x7fe8140a41b0 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3256346173 0 0) 0x7fe7fc003520 con 0x7fe814137940 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 >> v1:172.21.15.81:6790/0 conn(0x7fe8140a41b0 legacy=0x7fe814140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 >> v1:172.21.15.81:6789/0 conn(0x7fe8140a49a0 legacy=0x7fe81413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 monclient: found mon.noname-c 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8140a39f0 con 0x7fe814137940 2024-05-08T15:49:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe804001730 con 0x7fe814137940 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: dump: 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.884 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 5 monclient: authenticate success, global_id 4936 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe804001990 con 0x7fe814137940 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 1 -- 172.21.15.7:0/2107046542 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe804004be0 con 0x7fe814137940 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe812ffd640 10 monclient: dump: 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 4 set_mon_vals no callback set 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 10 monclient: shutdown 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2107046542 >> v1:172.21.15.7:6791/0 conn(0x7fe814137940 legacy=0x7fe814137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2107046542 shutdown_connections 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2107046542 >> 172.21.15.7:0/2107046542 conn(0x7fe8140a9fa0 msgr2=0x7fe8140aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2107046542 shutdown_connections 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.875+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2107046542 wait complete. 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.876+0000 7fe81a2c8640 10 monclient: build_initial_monmap 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.876+0000 7fe81a2c8640 10 monclient: monmap: 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:14.888 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.876+0000 7fe81a2c8640 1 Processor -- start 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.876+0000 7fe81a2c8640 1 -- start start 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.876+0000 7fe81a2c8640 10 monclient: init 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 10 monclient: _add_conns ranks=[1,2,4,6,8,0,7,5,3] 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 10 monclient(hunting): picked mon.noname-f con 0x7fe8140a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 10 monclient(hunting): picked mon.noname-b con 0x7fe8140a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 10 monclient(hunting): picked mon.noname-c con 0x7fe814137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe814132fa0 con 0x7fe8140a41b0 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe814135690 con 0x7fe8140a49a0 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe814135970 con 0x7fe814137940 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe81a2c8640 10 monclient(hunting): authenticate will time out at 962.992946s 2024-05-08T15:49:14.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe8137fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fe8140a49a0 0x7fe814132890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43528/0 (socket says 172.21.15.7:43528) 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe818865640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fe814137940 0x7fe8141d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57810/0 (socket says 172.21.15.7:57810) 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe8137fe640 1 -- 172.21.15.7:0/2010435686 learned_addr learned my addr 172.21.15.7:0/2010435686 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 264730397 0 0) 0x7fe814135690 con 0x7fe8140a49a0 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): my global_id is 94106 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe800003520 con 0x7fe8140a49a0 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798796280 0 0) 0x7fe814135970 con 0x7fe814137940 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): my global_id is 4945 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe814135690 con 0x7fe814137940 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2673739352 0 0) 0x7fe814132fa0 con 0x7fe8140a41b0 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): my global_id is 4906 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe814135970 con 0x7fe8140a41b0 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2304757606 0 0) 0x7fe800003520 con 0x7fe8140a49a0 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe814132fa0 con 0x7fe8140a49a0 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4285909364 0 0) 0x7fe814135690 con 0x7fe814137940 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe800003520 con 0x7fe814137940 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe808001a90 con 0x7fe8140a49a0 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:14.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.877+0000 7fe810ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.892 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe80c001dd0 con 0x7fe814137940 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.893 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4098453433 0 0) 0x7fe814135970 con 0x7fe8140a41b0 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe814135690 con 0x7fe8140a41b0 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe8040016b0 con 0x7fe8140a41b0 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:14.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 599431370 0 0) 0x7fe814132fa0 con 0x7fe8140a49a0 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.7:6791/0 conn(0x7fe814137940 legacy=0x7fe8141d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.81:6789/0 conn(0x7fe8140a41b0 legacy=0x7fe814132180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 monclient: found mon.b 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8141353b0 con 0x7fe8140a49a0 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: _finish_auth 0 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe81a2c8640 5 monclient: authenticate success, global_id 94106 2024-05-08T15:49:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe81a2c8640 10 monclient: _renew_subs 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe81a2c8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe8141363e0 con 0x7fe8140a49a0 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe81a2c8640 10 monclient: _renew_subs 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe81a2c8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fe814133e40 con 0x7fe8140a49a0 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe8080038c0 con 0x7fe8140a49a0 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe808001c70 con 0x7fe8140a49a0 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe8127fc640 4 set_mon_vals no callback set 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.878+0000 7fe810ff9640 10 monclient: dump: 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.897 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fe808003ef0 con 0x7fe8140a49a0 2024-05-08T15:49:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe81a2c8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe7f40065c0 con 0x7fe7f4002860 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(96..96 src has 1..96) ==== 3506+0+0 (unknown 3001059884 0 0) 0x7fe808046d30 con 0x7fe8140a49a0 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4153573807 0 0) 0x7fe7f40065c0 con 0x7fe7f4002860 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 10 monclient(hunting): my global_id is 94104 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe814132fa0 con 0x7fe7f4002860 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe818865640 10 monclient: get_auth_request con 0x7fe80003a450 auth_method 0 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3819474536 0 0) 0x7fe814132fa0 con 0x7fe7f4002860 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7f40065c0 con 0x7fe7f4002860 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe804001990 con 0x7fe7f4002860 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.879+0000 7fe810ff9640 10 monclient: dump: 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.880+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3788912905 0 0) 0x7fe7f40065c0 con 0x7fe7f4002860 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.880+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7f400a150 con 0x7fe7f4002860 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.880+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe804005010 con 0x7fe7f4002860 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.880+0000 7fe810ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.880+0000 7fe810ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.880+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.7:6789/0 conn(0x7fe7f4002860 legacy=0x7fe7f4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.880+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.7:6789/0 conn(0x7fe7f4002860 legacy=0x7fe7f4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.886+0000 7fe81a2c8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.886+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe7f4009e20 con 0x7fe7f40090e0 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218917976 0 0) 0x7fe7f4009e20 con 0x7fe7f40090e0 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient(hunting): my global_id is 94113 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7f40065c0 con 0x7fe7f40090e0 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4198478329 0 0) 0x7fe7f40065c0 con 0x7fe7f40090e0 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe7f4009e20 con 0x7fe7f40090e0 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe804005770 con 0x7fe7f40090e0 2024-05-08T15:49:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient: dump: 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3476598408 0 0) 0x7fe7f4009e20 con 0x7fe7f40090e0 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe7f400c390 con 0x7fe7f40090e0 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 3282168416) 0x7fe804006c00 con 0x7fe7f40090e0 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.7:6789/0 conn(0x7fe7f40090e0 legacy=0x7fe7f4009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.887+0000 7fe810ff9640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.7:6789/0 conn(0x7fe7f40090e0 legacy=0x7fe7f4009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:14.905 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 5438, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:14.908 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:14.909 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:14.910 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.911 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.912 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.913 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.914 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.915 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.916 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.917 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.918 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.919 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:14.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:14.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.922 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:14.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:14.924 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:14.924 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.81:6812/847627118 conn(0x7fe80003a450 legacy=0x7fe80003c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 10 monclient: shutdown 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 >> v1:172.21.15.7:6790/0 conn(0x7fe8140a49a0 legacy=0x7fe814132890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe818865640 1 -- 172.21.15.7:0/2010435686 reap_dead start 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe818865640 1 -- 172.21.15.7:0/2010435686 reap_dead start 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 shutdown_connections 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 >> 172.21.15.7:0/2010435686 conn(0x7fe8140a9fa0 msgr2=0x7fe814130bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:14.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 shutdown_connections 2024-05-08T15:49:14.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:14.888+0000 7fe81a2c8640 1 -- 172.21.15.7:0/2010435686 wait complete. 2024-05-08T15:49:14.926 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:14.926 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:49:14.926 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.032+0000 7f244667b640 10 monclient: get_monmap_and_config 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.032+0000 7f244667b640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.032+0000 7f244667b640 10 monclient: monmap: 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:15.034726+0000 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:15.034726+0000 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.031 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.033+0000 7f244667b640 1 Processor -- start 2024-05-08T15:49:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.033+0000 7f244667b640 1 -- start start 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.033+0000 7f244667b640 10 monclient: init 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.033+0000 7f244667b640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.033+0000 7f244667b640 10 monclient: _add_conns ranks=[5,0,3,7,8,1,2,6,4] 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.033+0000 7f244667b640 10 monclient(hunting): picked mon.noname-h con 0x7f2440137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.033+0000 7f244667b640 10 monclient(hunting): picked mon.noname-a con 0x7f24400a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 10 monclient(hunting): picked mon.noname-g con 0x7f24400a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24400a2c70 con 0x7f24400a49a0 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24400a2f50 con 0x7f24400a41b0 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24400a1670 con 0x7f2440137940 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 10 monclient(hunting): authenticate will time out at 963.149652s 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243f7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f24400a49a0 0x7f244013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48800/0 (socket says 172.21.15.7:48800) 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243f7fe640 1 -- 172.21.15.7:0/3145001018 learned_addr learned my addr 172.21.15.7:0/3145001018 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1182228875 0 0) 0x7f24400a2c70 con 0x7f24400a49a0 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): my global_id is 94122 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2428003520 con 0x7f24400a49a0 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2262341284 0 0) 0x7f24400a2f50 con 0x7f24400a41b0 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): my global_id is 75036 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24400a2c70 con 0x7f24400a41b0 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3825477692 0 0) 0x7f24400a1670 con 0x7f2440137940 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient(hunting): my global_id is 4946 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24400a2f50 con 0x7f2440137940 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2042241586 0 0) 0x7f2428003520 con 0x7f24400a49a0 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 >> v1:172.21.15.81:6791/0 conn(0x7f2440137940 legacy=0x7f2440137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 >> v1:172.21.15.81:6790/0 conn(0x7f24400a41b0 legacy=0x7f2440140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 monclient: found mon.noname-a 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24400a39f0 con 0x7f24400a49a0 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f24340017b0 con 0x7f24400a49a0 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 5 monclient: authenticate success, global_id 94122 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: dump: 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2434001a10 con 0x7f24400a49a0 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 1 -- 172.21.15.7:0/3145001018 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2434004be0 con 0x7f24400a49a0 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f244667b640 4 set_mon_vals no callback set 2024-05-08T15:49:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.034+0000 7f243effd640 10 monclient: dump: 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 10 monclient: shutdown 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 1 -- 172.21.15.7:0/3145001018 >> v1:172.21.15.7:6789/0 conn(0x7f24400a49a0 legacy=0x7f244013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 1 -- 172.21.15.7:0/3145001018 shutdown_connections 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 1 -- 172.21.15.7:0/3145001018 >> 172.21.15.7:0/3145001018 conn(0x7f24400a9fa0 msgr2=0x7f24400aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 1 -- 172.21.15.7:0/3145001018 shutdown_connections 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 1 -- 172.21.15.7:0/3145001018 wait complete. 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.035+0000 7f244667b640 10 monclient: monmap: 2024-05-08T15:49:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.047 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 1 Processor -- start 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 1 -- start start 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient: init 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient: _add_conns ranks=[2,1,8,0,4,5,7,6,3] 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient(hunting): picked mon.noname-b con 0x7f24400a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient(hunting): picked mon.noname-f con 0x7f24400a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient(hunting): picked mon.noname-e con 0x7f2440137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2440136180 con 0x7f24400a49a0 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24400a1f30 con 0x7f24400a41b0 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2440135a80 con 0x7f2440137940 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.036+0000 7f244667b640 10 monclient(hunting): authenticate will time out at 963.152537s 2024-05-08T15:49:15.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243ffff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f24400a41b0 0x7f2440131f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43536/0 (socket says 172.21.15.7:43536) 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243ffff640 1 -- 172.21.15.7:0/473220019 learned_addr learned my addr 172.21.15.7:0/473220019 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496914402 0 0) 0x7f2440135a80 con 0x7f2440137940 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): my global_id is 94112 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f242c003520 con 0x7f2440137940 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 262646855 0 0) 0x7f24400a2f50 con 0x7f24400a41b0 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): my global_id is 94115 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2440135a80 con 0x7f24400a41b0 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2211221620 0 0) 0x7f24400a39f0 con 0x7f24400a49a0 2024-05-08T15:49:15.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): my global_id is 4915 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24400a2f50 con 0x7f24400a49a0 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1701463992 0 0) 0x7f2440135a80 con 0x7f24400a41b0 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f24400a39f0 con 0x7f24400a41b0 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2065320181 0 0) 0x7f242c003520 con 0x7f2440137940 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2440135a80 con 0x7f2440137940 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f24300019f0 con 0x7f24400a41b0 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): dump: 2024-05-08T15:49:15.050 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.051 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2575453155 0 0) 0x7f24400a2f50 con 0x7f24400a49a0 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f242c003520 con 0x7f24400a49a0 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2438001d50 con 0x7f2440137940 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): dump: 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.052 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2434004b20 con 0x7f24400a49a0 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient(hunting): dump: 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.054 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 473071977 0 0) 0x7f24400a39f0 con 0x7f24400a41b0 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.7:6793/0 conn(0x7f2440137940 legacy=0x7f24401d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.81:6789/0 conn(0x7f24400a49a0 legacy=0x7f2440132630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 monclient: found mon.b 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24400a0f60 con 0x7f24400a41b0 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f244667b640 5 monclient: authenticate success, global_id 94115 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f244667b640 10 monclient: _renew_subs 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f244667b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2440135000 con 0x7f24400a41b0 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f244667b640 10 monclient: _renew_subs 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f244667b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f24401357a0 con 0x7f24400a41b0 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f24300038c0 con 0x7f24400a41b0 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2430003e30 con 0x7f24400a41b0 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243e7fc640 4 set_mon_vals no callback set 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.037+0000 7f243cff9640 10 monclient: dump: 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.056 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.057 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.038+0000 7f244667b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.038+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2430004050 con 0x7f24400a41b0 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.038+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f24200065c0 con 0x7f2420002860 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(96..96 src has 1..96) ==== 3506+0+0 (unknown 3001059884 0 0) 0x7f2430046d20 con 0x7f24400a41b0 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3282578664 0 0) 0x7f24200065c0 con 0x7f2420002860 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient(hunting): my global_id is 94124 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24400a39f0 con 0x7f2420002860 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f2444c18640 10 monclient: get_auth_request con 0x7f242c03a390 auth_method 0 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 670688247 0 0) 0x7f24400a39f0 con 0x7f2420002860 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f24200065c0 con 0x7f2420002860 2024-05-08T15:49:15.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f24340010d0 con 0x7f2420002860 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient: dump: 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.059 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2834984017 0 0) 0x7f24200065c0 con 0x7f2420002860 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f242000a150 con 0x7f2420002860 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2434007480 con 0x7f2420002860 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.7:6790/0 conn(0x7f2420002860 legacy=0x7f2420004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.039+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.7:6790/0 conn(0x7f2420002860 legacy=0x7f2420004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f244667b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2420009e40 con 0x7f2420009100 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2576372523 0 0) 0x7f2420009e40 con 0x7f2420009100 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 10 monclient(hunting): my global_id is 94133 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f24200065c0 con 0x7f2420009100 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4139246199 0 0) 0x7f24200065c0 con 0x7f2420009100 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2420009e40 con 0x7f2420009100 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2434005630 con 0x7f2420009100 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:15.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 10 monclient: dump: 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.063 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1850498123 0 0) 0x7f2420009e40 con 0x7f2420009100 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.046+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f242000c3d0 con 0x7f2420009100 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 3340952657) 0x7f2434007630 con 0x7f2420009100 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f243cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f243cff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.7:6790/0 conn(0x7f2420009100 legacy=0x7f2420009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f243cff9640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.7:6790/0 conn(0x7f2420009100 legacy=0x7f2420009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 5597, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:15.068 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.069 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:15.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:15.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.073 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:15.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:15.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.079 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.080 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:15.081 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:15.082 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.81:6812/847627118 conn(0x7f242c03a390 legacy=0x7f242c03c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 10 monclient: shutdown 2024-05-08T15:49:15.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 >> v1:172.21.15.7:6790/0 conn(0x7f24400a41b0 legacy=0x7f2440131f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f2444c18640 1 -- 172.21.15.7:0/473220019 reap_dead start 2024-05-08T15:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f2444c18640 1 -- 172.21.15.7:0/473220019 reap_dead start 2024-05-08T15:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 shutdown_connections 2024-05-08T15:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 >> 172.21.15.7:0/473220019 conn(0x7f24400a9fa0 msgr2=0x7f2440130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 shutdown_connections 2024-05-08T15:49:15.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.047+0000 7f244667b640 1 -- 172.21.15.7:0/473220019 wait complete. 2024-05-08T15:49:15.084 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:15.084 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:49:15.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:49:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.191+0000 7f43384d9640 10 monclient: get_monmap_and_config 2024-05-08T15:49:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.191+0000 7f43384d9640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.191+0000 7f43384d9640 10 monclient: monmap: 2024-05-08T15:49:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:15.193033+0000 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:15.193033+0000 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.190 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.191+0000 7f43384d9640 1 Processor -- start 2024-05-08T15:49:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.191+0000 7f43384d9640 1 -- start start 2024-05-08T15:49:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.191+0000 7f43384d9640 10 monclient: init 2024-05-08T15:49:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 10 monclient: _add_conns ranks=[4,0,1,3,5,7,8,6,2] 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 10 monclient(hunting): picked mon.noname-c con 0x7f43300a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 10 monclient(hunting): picked mon.noname-a con 0x7f43300a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 10 monclient(hunting): picked mon.noname-f con 0x7f433013e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300a1660 con 0x7f43300a5780 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300a28f0 con 0x7f433013e310 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300a2bd0 con 0x7f43300a53a0 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f43384d9640 10 monclient(hunting): authenticate will time out at 963.307998s 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4336275640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f43300a53a0 0x7f43300a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57818/0 (socket says 172.21.15.7:57818) 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335a74640 1 --1- 172.21.15.7:0/4163512337 >> v1:172.21.15.7:6789/0 conn(0x7f43300a5780 0x7f433013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48806/0 (socket says 172.21.15.7:48806) 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4336275640 1 -- 172.21.15.7:0/4163512337 learned_addr learned my addr 172.21.15.7:0/4163512337 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977220927 0 0) 0x7f43300a2bd0 con 0x7f43300a53a0 2024-05-08T15:49:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): my global_id is 4954 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f431c003520 con 0x7f43300a53a0 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 483226178 0 0) 0x7f43300a1660 con 0x7f43300a5780 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): my global_id is 94131 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43300a2bd0 con 0x7f43300a5780 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972156397 0 0) 0x7f43300a28f0 con 0x7f433013e310 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 10 monclient(hunting): my global_id is 4924 2024-05-08T15:49:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.192+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43300a1660 con 0x7f433013e310 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3428155122 0 0) 0x7f43300a2bd0 con 0x7f43300a5780 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 >> v1:172.21.15.7:6791/0 conn(0x7f43300a53a0 legacy=0x7f43300a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 >> v1:172.21.15.81:6789/0 conn(0x7f433013e310 legacy=0x7f4330140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 monclient: found mon.noname-a 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43300a6730 con 0x7f43300a5780 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4320001710 con 0x7f43300a5780 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 5 monclient: authenticate success, global_id 94131 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: dump: 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f43200019f0 con 0x7f43300a5780 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 1 -- 172.21.15.7:0/4163512337 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4320004b20 con 0x7f43300a5780 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 4 set_mon_vals no callback set 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f4335273640 10 monclient: dump: 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 10 monclient: shutdown 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 1 -- 172.21.15.7:0/4163512337 >> v1:172.21.15.7:6789/0 conn(0x7f43300a5780 legacy=0x7f433013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 1 -- 172.21.15.7:0/4163512337 shutdown_connections 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 1 -- 172.21.15.7:0/4163512337 >> 172.21.15.7:0/4163512337 conn(0x7f43300a9fa0 msgr2=0x7f43300aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 1 -- 172.21.15.7:0/4163512337 shutdown_connections 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.193+0000 7f43384d9640 1 -- 172.21.15.7:0/4163512337 wait complete. 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.194+0000 7f43384d9640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.194+0000 7f43384d9640 10 monclient: monmap: 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.194+0000 7f43384d9640 1 Processor -- start 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.194+0000 7f43384d9640 1 -- start start 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.194+0000 7f43384d9640 10 monclient: init 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient: _add_conns ranks=[4,3,5,2,7,0,8,1,6] 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient(hunting): picked mon.noname-c con 0x7f43300a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient(hunting): picked mon.noname-g con 0x7f43300a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient(hunting): picked mon.noname-h con 0x7f433013e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43300a43c0 con 0x7f43300a5780 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4330135220 con 0x7f43300a53a0 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43301346c0 con 0x7f433013e310 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient(hunting): authenticate will time out at 963.310811s 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4336275640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f43300a53a0 0x7f43301cc2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57832/0 (socket says 172.21.15.7:57832) 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4336275640 1 -- 172.21.15.7:0/1658277451 learned_addr learned my addr 172.21.15.7:0/1658277451 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030662891 0 0) 0x7f4330135220 con 0x7f43300a53a0 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): my global_id is 4963 2024-05-08T15:49:15.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4314003520 con 0x7f43300a53a0 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1139832269 0 0) 0x7f4314003520 con 0x7f43300a53a0 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4330135220 con 0x7f43300a53a0 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f432c001990 con 0x7f43300a53a0 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1028930259 0 0) 0x7f43300a43c0 con 0x7f43300a5780 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): my global_id is 75045 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4314003520 con 0x7f43300a5780 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897041797 0 0) 0x7f43301346c0 con 0x7f433013e310 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient(hunting): my global_id is 4955 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43300a43c0 con 0x7f433013e310 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 709492666 0 0) 0x7f4330135220 con 0x7f43300a53a0 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.81:6791/0 conn(0x7f433013e310 legacy=0x7f43301d4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.81:6790/0 conn(0x7f43300a5780 legacy=0x7f4330137ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 monclient: found mon.c 2024-05-08T15:49:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43301366d0 con 0x7f43300a53a0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f4326ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 5 monclient: authenticate success, global_id 4963 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient: _renew_subs 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.195+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4330135460 con 0x7f43300a53a0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f43384d9640 10 monclient: _renew_subs 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f43384d9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f43301d5d30 con 0x7f43300a53a0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f432c004e10 con 0x7f43300a53a0 2024-05-08T15:49:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4326ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f432c005300 con 0x7f43300a53a0 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4326ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4334a72640 4 set_mon_vals no callback set 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4326ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4326ffd640 10 monclient: dump: 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.213 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f43384d9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f43180065c0 con 0x7f4318002860 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.196+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f432c010680 con 0x7f43300a53a0 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(96..96 src has 1..96) ==== 3506+0+0 (unknown 3001059884 0 0) 0x7f432c047210 con 0x7f43300a53a0 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1813451985 0 0) 0x7f43180065c0 con 0x7f4318002860 2024-05-08T15:49:15.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient(hunting): my global_id is 4972 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4330135220 con 0x7f4318002860 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4336a76640 10 monclient: get_auth_request con 0x7f4314039cf0 auth_method 0 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4047380341 0 0) 0x7f4330135220 con 0x7f4318002860 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43180065c0 con 0x7f4318002860 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4320004b20 con 0x7f4318002860 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient: dump: 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2061659409 0 0) 0x7f43180065c0 con 0x7f4318002860 2024-05-08T15:49:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f431800a150 con 0x7f4318002860 2024-05-08T15:49:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4320001100 con 0x7f4318002860 2024-05-08T15:49:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.7:6791/0 conn(0x7f4318002860 legacy=0x7f4318004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.197+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.7:6791/0 conn(0x7f4318002860 legacy=0x7f4318004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f43384d9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4318009e20 con 0x7f43180090e0 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4107144756 0 0) 0x7f4318009e20 con 0x7f43180090e0 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 10 monclient(hunting): my global_id is 4981 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43180065c0 con 0x7f43180090e0 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3883233359 0 0) 0x7f43180065c0 con 0x7f43180090e0 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4318009e20 con 0x7f43180090e0 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4320005000 con 0x7f43180090e0 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.204+0000 7f4326ffd640 10 monclient: dump: 2024-05-08T15:49:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1114392355 0 0) 0x7f4318009e20 con 0x7f43180090e0 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f431800c390 con 0x7f43180090e0 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 2137619905) 0x7f43200072e0 con 0x7f43180090e0 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4326ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4326ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.7:6791/0 conn(0x7f43180090e0 legacy=0x7f4318009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4326ffd640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.7:6791/0 conn(0x7f43180090e0 legacy=0x7f4318009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 147513, 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.233 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:15.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.237 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.239 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.240 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.241 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:15.241 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.81:6812/847627118 conn(0x7f4314039cf0 legacy=0x7f431403c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 10 monclient: shutdown 2024-05-08T15:49:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 >> v1:172.21.15.7:6791/0 conn(0x7f43300a53a0 legacy=0x7f43301cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4336a76640 1 -- 172.21.15.7:0/1658277451 reap_dead start 2024-05-08T15:49:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f4336a76640 1 -- 172.21.15.7:0/1658277451 reap_dead start 2024-05-08T15:49:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 shutdown_connections 2024-05-08T15:49:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 >> 172.21.15.7:0/1658277451 conn(0x7f43300a9fa0 msgr2=0x7f43300a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 shutdown_connections 2024-05-08T15:49:15.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.205+0000 7f43384d9640 1 -- 172.21.15.7:0/1658277451 wait complete. 2024-05-08T15:49:15.243 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:15.243 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:49:15.243 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:49:15.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.349+0000 7f91251fb640 10 monclient: get_monmap_and_config 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.349+0000 7f91251fb640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.349+0000 7f91251fb640 10 monclient: monmap: 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:15.351645+0000 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:15.351645+0000 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.347 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.348 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 1 Processor -- start 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 1 -- start start 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient: init 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient: _add_conns ranks=[6,3,8,4,0,1,5,2,7] 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient(hunting): picked mon.noname-d con 0x7f9120137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient(hunting): picked mon.noname-g con 0x7f91200a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient(hunting): picked mon.noname-e con 0x7f91200a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91200a2c70 con 0x7f91200a49a0 2024-05-08T15:49:15.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91200a2f50 con 0x7f9120137940 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91200a1670 con 0x7f91200a41b0 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f91251fb640 10 monclient(hunting): authenticate will time out at 963.466470s 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f911ed9d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f9120137940 0x7f9120137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35468/0 (socket says 172.21.15.7:35468) 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911ed9d640 1 -- 172.21.15.7:0/2886378921 learned_addr learned my addr 172.21.15.7:0/2886378921 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.350+0000 7f911f59e640 1 --1- 172.21.15.7:0/2886378921 >> v1:172.21.15.7:6793/0 conn(0x7f91200a41b0 0x7f9120140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:57792/0 (socket says 172.21.15.7:57792) 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3735821980 0 0) 0x7f91200a2f50 con 0x7f9120137940 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): my global_id is 74985 2024-05-08T15:49:15.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9108003520 con 0x7f9120137940 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 540151718 0 0) 0x7f91200a1670 con 0x7f91200a41b0 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): my global_id is 94121 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91200a2f50 con 0x7f91200a41b0 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3516100142 0 0) 0x7f91200a2c70 con 0x7f91200a49a0 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient(hunting): my global_id is 75054 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91200a1670 con 0x7f91200a49a0 2024-05-08T15:49:15.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2496533180 0 0) 0x7f91200a2f50 con 0x7f91200a41b0 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 >> v1:172.21.15.7:6792/0 conn(0x7f9120137940 legacy=0x7f9120137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 >> v1:172.21.15.81:6790/0 conn(0x7f91200a49a0 legacy=0x7f912013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 monclient: found mon.noname-e 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91200a39f0 con 0x7f91200a41b0 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9118001b10 con 0x7f91200a41b0 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f91251fb640 5 monclient: authenticate success, global_id 94121 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f91251fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: dump: 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.363 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f91251fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9118001df0 con 0x7f91200a41b0 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 1 -- 172.21.15.7:0/2886378921 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9118004ea0 con 0x7f91200a41b0 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f911dd9b640 10 monclient: dump: 2024-05-08T15:49:15.364 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.365 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f91251fb640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.351+0000 7f91251fb640 4 set_mon_vals no callback set 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 10 monclient: shutdown 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 1 -- 172.21.15.7:0/2886378921 >> v1:172.21.15.7:6793/0 conn(0x7f91200a41b0 legacy=0x7f9120140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 1 -- 172.21.15.7:0/2886378921 shutdown_connections 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 1 -- 172.21.15.7:0/2886378921 >> 172.21.15.7:0/2886378921 conn(0x7f91200a9fa0 msgr2=0x7f91200aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 1 -- 172.21.15.7:0/2886378921 shutdown_connections 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 1 -- 172.21.15.7:0/2886378921 wait complete. 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.352+0000 7f91251fb640 10 monclient: monmap: 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.366 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 1 Processor -- start 2024-05-08T15:49:15.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 1 -- start start 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 10 monclient: init 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 10 monclient: _add_conns ranks=[3,6,0,2,7,1,5,8,4] 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 10 monclient(hunting): picked mon.noname-g con 0x7f91200a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 10 monclient(hunting): picked mon.noname-d con 0x7f91200a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 10 monclient(hunting): picked mon.noname-a con 0x7f9120137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91200a1f30 con 0x7f9120137940 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91200a0f60 con 0x7f91200a41b0 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9120138c10 con 0x7f91200a49a0 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.353+0000 7f91251fb640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 10 monclient(hunting): authenticate will time out at 963.469611s 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f911e59c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f91200a49a0 0x7f91201d15f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35478/0 (socket says 172.21.15.7:35478) 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f911e59c640 1 -- 172.21.15.7:0/2721932981 learned_addr learned my addr 172.21.15.7:0/2721932981 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2744305862 0 0) 0x7f9120138c10 con 0x7f91200a49a0 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): my global_id is 74994 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f910c003520 con 0x7f91200a49a0 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1258752180 0 0) 0x7f91200a1f30 con 0x7f9120137940 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): my global_id is 94140 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9120138c10 con 0x7f9120137940 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182530386 0 0) 0x7f91200a0f60 con 0x7f91200a41b0 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): my global_id is 75063 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91200a1f30 con 0x7f91200a41b0 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2674331012 0 0) 0x7f910c003520 con 0x7f91200a49a0 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91200a0f60 con 0x7f91200a49a0 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9114001ad0 con 0x7f91200a49a0 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): dump: 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.370 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 455560524 0 0) 0x7f91200a1f30 con 0x7f91200a41b0 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f910c003520 con 0x7f91200a41b0 2024-05-08T15:49:15.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f91100018a0 con 0x7f91200a41b0 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): dump: 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.372 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3188571009 0 0) 0x7f9120138c10 con 0x7f9120137940 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91200a1f30 con 0x7f9120137940 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9118001ad0 con 0x7f9120137940 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient(hunting): dump: 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.373 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1900767176 0 0) 0x7f91200a0f60 con 0x7f91200a49a0 2024-05-08T15:49:15.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.81:6790/0 conn(0x7f91200a41b0 legacy=0x7f91201411b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.7:6789/0 conn(0x7f9120137940 legacy=0x7f91201d4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 monclient: found mon.d 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9120136100 con 0x7f91200a49a0 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 5 monclient: authenticate success, global_id 74994 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 10 monclient: _renew_subs 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f91201418c0 con 0x7f91200a49a0 2024-05-08T15:49:15.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 10 monclient: _renew_subs 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f91201367e0 con 0x7f91200a49a0 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f91140038c0 con 0x7f91200a49a0 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9114003e30 con 0x7f91200a49a0 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f91077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.354+0000 7f911d59a640 4 set_mon_vals no callback set 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.355+0000 7f91077fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.355+0000 7f91077fe640 10 monclient: dump: 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.376 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.355+0000 7f91251fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.355+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f9114003fd0 con 0x7f91200a49a0 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.355+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f91000065c0 con 0x7f9100002860 2024-05-08T15:49:15.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.355+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(96..96 src has 1..96) ==== 3506+0+0 (unknown 3001059884 0 0) 0x7f9114046d80 con 0x7f91200a49a0 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4207304322 0 0) 0x7f91000065c0 con 0x7f9100002860 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient(hunting): my global_id is 75003 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91200a0f60 con 0x7f9100002860 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f911f59e640 10 monclient: get_auth_request con 0x7f910c03a4c0 auth_method 0 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1799322587 0 0) 0x7f91200a0f60 con 0x7f9100002860 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91000065c0 con 0x7f9100002860 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9110004b20 con 0x7f9100002860 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:15.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient: dump: 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.379 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2397061385 0 0) 0x7f91000065c0 con 0x7f9100002860 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f910000a150 con 0x7f9100002860 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9110006060 con 0x7f9100002860 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.7:6792/0 conn(0x7f9100002860 legacy=0x7f9100004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.356+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.7:6792/0 conn(0x7f9100002860 legacy=0x7f9100004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91251fb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9100009e40 con 0x7f9100009100 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3292600630 0 0) 0x7f9100009e40 con 0x7f9100009100 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 10 monclient(hunting): my global_id is 75012 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91000065c0 con 0x7f9100009100 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1404181540 0 0) 0x7f91000065c0 con 0x7f9100009100 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9100009e40 con 0x7f9100009100 2024-05-08T15:49:15.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9110006890 con 0x7f9100009100 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 10 monclient: dump: 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.383 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 946642787 0 0) 0x7f9100009e40 con 0x7f9100009100 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.363+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f910000c3d0 con 0x7f9100009100 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7863 (unknown 0 0 4059373564) 0x7f9110006bb0 con 0x7f9100009100 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91077fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91077fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:15.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.7:6792/0 conn(0x7f9100009100 legacy=0x7f9100009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 147678, 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:15.385 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.386 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.387 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:15.388 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.389 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.390 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.391 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:15.392 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:15.393 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.394 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.397 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:15.398 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.399 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.400 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:15.402 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.403 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:15.404 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91077fe640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.7:6792/0 conn(0x7f9100009100 legacy=0x7f9100009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.81:6812/847627118 conn(0x7f910c03a4c0 legacy=0x7f910c03c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 10 monclient: shutdown 2024-05-08T15:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 >> v1:172.21.15.7:6792/0 conn(0x7f91200a49a0 legacy=0x7f91201d15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f911f59e640 1 -- 172.21.15.7:0/2721932981 reap_dead start 2024-05-08T15:49:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f911f59e640 1 -- 172.21.15.7:0/2721932981 reap_dead start 2024-05-08T15:49:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 shutdown_connections 2024-05-08T15:49:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 >> 172.21.15.7:0/2721932981 conn(0x7f91200a9fa0 msgr2=0x7f91201307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 shutdown_connections 2024-05-08T15:49:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.364+0000 7f91251fb640 1 -- 172.21.15.7:0/2721932981 wait complete. 2024-05-08T15:49:15.407 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:15.407 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:49:15.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:49:15.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.521+0000 7fb48c4d2640 10 monclient: get_monmap_and_config 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.521+0000 7fb48c4d2640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.521+0000 7fb48c4d2640 10 monclient: monmap: 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:15.523349+0000 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:15.523349+0000 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.519 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.520 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.520 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.522+0000 7fb48c4d2640 1 Processor -- start 2024-05-08T15:49:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.522+0000 7fb48c4d2640 1 -- start start 2024-05-08T15:49:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.522+0000 7fb48c4d2640 10 monclient: init 2024-05-08T15:49:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 10 monclient: _add_conns ranks=[2,1,7,6,0,4,3,5,8] 2024-05-08T15:49:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 10 monclient(hunting): picked mon.noname-b con 0x7fb484072300 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 10 monclient(hunting): picked mon.noname-f con 0x7fb48413c070 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 10 monclient(hunting): picked mon.noname-i con 0x7fb48413c4c0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb484072df0 con 0x7fb48413c070 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb484070300 con 0x7fb484072300 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4840705e0 con 0x7fb48413c4c0 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.523+0000 7fb48c4d2640 10 monclient(hunting): authenticate will time out at 963.639453s 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48aa6f640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7fb48413c4c0 0x7fb484144600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:50936/0 (socket says 172.21.15.7:50936) 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48aa6f640 1 -- 172.21.15.7:0/3567541536 learned_addr learned my addr 172.21.15.7:0/3567541536 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb489a6d640 1 --1- 172.21.15.7:0/3567541536 >> v1:172.21.15.81:6789/0 conn(0x7fb48413c070 0x7fb484140ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:43408/0 (socket says 172.21.15.7:43408) 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 520534110 0 0) 0x7fb484070300 con 0x7fb484072300 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): my global_id is 94142 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb470003520 con 0x7fb484072300 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 953296292 0 0) 0x7fb4840705e0 con 0x7fb48413c4c0 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): my global_id is 94120 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb484070300 con 0x7fb48413c4c0 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2585060682 0 0) 0x7fb484072df0 con 0x7fb48413c070 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient(hunting): my global_id is 4933 2024-05-08T15:49:15.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4840705e0 con 0x7fb48413c070 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3052410412 0 0) 0x7fb470003520 con 0x7fb484072300 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 >> v1:172.21.15.81:6792/0 conn(0x7fb48413c4c0 legacy=0x7fb484144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 >> v1:172.21.15.81:6789/0 conn(0x7fb48413c070 legacy=0x7fb484140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 monclient: found mon.noname-b 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:49:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48406efe0 con 0x7fb484072300 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4800017b0 con 0x7fb484072300 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48c4d2640 5 monclient: authenticate success, global_id 94142 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: dump: 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48c4d2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb480001a60 con 0x7fb484072300 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 1 -- 172.21.15.7:0/3567541536 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb480003950 con 0x7fb484072300 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48c4d2640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48c4d2640 4 set_mon_vals no callback set 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.524+0000 7fb48926c640 10 monclient: dump: 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.525+0000 7fb48c4d2640 10 monclient: shutdown 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.525+0000 7fb48c4d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.525+0000 7fb48c4d2640 1 -- 172.21.15.7:0/3567541536 >> v1:172.21.15.7:6790/0 conn(0x7fb484072300 legacy=0x7fb4840726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.525+0000 7fb48c4d2640 1 -- 172.21.15.7:0/3567541536 shutdown_connections 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.525+0000 7fb48c4d2640 1 -- 172.21.15.7:0/3567541536 >> 172.21.15.7:0/3567541536 conn(0x7fb4840a6280 msgr2=0x7fb4840a2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.525+0000 7fb48c4d2640 1 -- 172.21.15.7:0/3567541536 shutdown_connections 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.525+0000 7fb48c4d2640 1 -- 172.21.15.7:0/3567541536 wait complete. 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient: monmap: 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.538 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 1 Processor -- start 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 1 -- start start 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient: init 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient: _add_conns ranks=[4,6,5,8,0,3,2,1,7] 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient(hunting): picked mon.noname-c con 0x7fb484072300 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient(hunting): picked mon.noname-d con 0x7fb48413c070 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 10 monclient(hunting): picked mon.noname-h con 0x7fb48413c4c0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.526+0000 7fb48c4d2640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb484137180 con 0x7fb484072300 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb48409e800 con 0x7fb48413c4c0 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb48409f140 con 0x7fb48413c070 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 10 monclient(hunting): authenticate will time out at 963.642666s 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48a26e640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fb484072300 0x7fb48409c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57866/0 (socket says 172.21.15.7:57866) 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48a26e640 1 -- 172.21.15.7:0/2523227200 learned_addr learned my addr 172.21.15.7:0/2523227200 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb489a6d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fb48413c070 0x7fb48409d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35504/0 (socket says 172.21.15.7:35504) 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096922826 0 0) 0x7fb48409f140 con 0x7fb48413c070 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): my global_id is 75021 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb468003520 con 0x7fb48413c070 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3840384436 0 0) 0x7fb484137180 con 0x7fb484072300 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): my global_id is 4990 2024-05-08T15:49:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb48409f140 con 0x7fb484072300 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1989226511 0 0) 0x7fb48409e800 con 0x7fb48413c4c0 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): my global_id is 4964 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb484137180 con 0x7fb48413c4c0 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2329447698 0 0) 0x7fb468003520 con 0x7fb48413c070 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb48409e800 con 0x7fb48413c070 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4740019d0 con 0x7fb48413c070 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): dump: 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.541 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4061719663 0 0) 0x7fb48409f140 con 0x7fb484072300 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb468003520 con 0x7fb484072300 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb480003930 con 0x7fb484072300 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): dump: 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.543 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3072487387 0 0) 0x7fb484137180 con 0x7fb48413c4c0 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb48409f140 con 0x7fb48413c4c0 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb47c001d10 con 0x7fb48413c4c0 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient(hunting): dump: 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.545 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3281187549 0 0) 0x7fb48409e800 con 0x7fb48413c070 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.81:6791/0 conn(0x7fb48413c4c0 legacy=0x7fb4841d8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.7:6791/0 conn(0x7fb484072300 legacy=0x7fb48409c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 monclient: found mon.d 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48409ec80 con 0x7fb48413c070 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb47affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 5 monclient: authenticate success, global_id 75021 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 10 monclient: _renew_subs 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb48409d7a0 con 0x7fb48413c070 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 10 monclient: _renew_subs 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.527+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fb4841da500 con 0x7fb48413c070 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb474004a90 con 0x7fb48413c070 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb47affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb474005000 con 0x7fb48413c070 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb47affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb47affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb488a6b640 4 set_mon_vals no callback set 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb47affd640 10 monclient: dump: 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.547 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb48c4d2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb46c0065c0 con 0x7fb46c002860 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.528+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fb4740051a0 con 0x7fb48413c070 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(97..97 src has 1..97) ==== 3898+0+0 (unknown 1278706646 0 0) 0x7fb474046db0 con 0x7fb48413c070 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025096071 0 0) 0x7fb46c0065c0 con 0x7fb46c002860 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient(hunting): my global_id is 94130 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb48409e800 con 0x7fb46c002860 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb48aa6f640 10 monclient: get_auth_request con 0x7fb46803a490 auth_method 0 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1432962728 0 0) 0x7fb48409e800 con 0x7fb46c002860 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb46c0065c0 con 0x7fb46c002860 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb480001870 con 0x7fb46c002860 2024-05-08T15:49:15.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient: dump: 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.550 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1184682834 0 0) 0x7fb46c0065c0 con 0x7fb46c002860 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb46c00a150 con 0x7fb46c002860 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb480004280 con 0x7fb46c002860 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.7:6793/0 conn(0x7fb46c002860 legacy=0x7fb46c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.529+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.7:6793/0 conn(0x7fb46c002860 legacy=0x7fb46c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb48c4d2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb46c009e50 con 0x7fb46c009110 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263027806 0 0) 0x7fb46c009e50 con 0x7fb46c009110 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 10 monclient(hunting): my global_id is 94139 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb46c0065c0 con 0x7fb46c009110 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1980354891 0 0) 0x7fb46c0065c0 con 0x7fb46c009110 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb46c009e50 con 0x7fb46c009110 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb480004430 con 0x7fb46c009110 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:15.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.536+0000 7fb47affd640 10 monclient: dump: 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2611848801 0 0) 0x7fb46c009e50 con 0x7fb46c009110 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb46c00c3d0 con 0x7fb46c009110 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 2507628399) 0x7fb480005de0 con 0x7fb46c009110 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb47affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb47affd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.7:6793/0 conn(0x7fb46c009110 legacy=0x7fb46c0095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb47affd640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.7:6793/0 conn(0x7fb46c009110 legacy=0x7fb46c0095a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:15.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 6089, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:15.556 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.557 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.558 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.559 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.560 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.561 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:15.562 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.564 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.567 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.568 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.569 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.570 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:15.571 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.572 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.574 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:15.574 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.81:6812/847627118 conn(0x7fb46803a490 legacy=0x7fb46803c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 10 monclient: shutdown 2024-05-08T15:49:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 >> v1:172.21.15.7:6792/0 conn(0x7fb48413c070 legacy=0x7fb48409d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48aa6f640 1 -- 172.21.15.7:0/2523227200 reap_dead start 2024-05-08T15:49:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48aa6f640 1 -- 172.21.15.7:0/2523227200 reap_dead start 2024-05-08T15:49:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 shutdown_connections 2024-05-08T15:49:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 >> 172.21.15.7:0/2523227200 conn(0x7fb4840a6280 msgr2=0x7fb4840a3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 shutdown_connections 2024-05-08T15:49:15.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.537+0000 7fb48c4d2640 1 -- 172.21.15.7:0/2523227200 wait complete. 2024-05-08T15:49:15.576 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:15.576 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:49:15.576 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:49:15.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.683+0000 7f767d5e6640 10 monclient: get_monmap_and_config 2024-05-08T15:49:15.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.683+0000 7f767d5e6640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.683+0000 7f767d5e6640 10 monclient: monmap: 2024-05-08T15:49:15.680 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:15.685353+0000 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:15.685353+0000 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.681 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.682 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.683+0000 7f767d5e6640 1 Processor -- start 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 1 -- start start 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient: init 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient: _add_conns ranks=[4,0,3,1,8,5,7,2,6] 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient(hunting): picked mon.noname-c con 0x7f7678137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient(hunting): picked mon.noname-a con 0x7f76780a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient(hunting): picked mon.noname-g con 0x7f76780a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780a2c70 con 0x7f76780a49a0 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780a2f50 con 0x7f76780a41b0 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780a1670 con 0x7f7678137940 2024-05-08T15:49:15.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f767d5e6640 10 monclient(hunting): authenticate will time out at 963.800142s 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f76767fc640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f76780a49a0 0x7f767813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48830/0 (socket says 172.21.15.7:48830) 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7676ffd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f7678137940 0x7f7678137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57882/0 (socket says 172.21.15.7:57882) 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f76767fc640 1 -- 172.21.15.7:0/480378275 learned_addr learned my addr 172.21.15.7:0/480378275 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960238678 0 0) 0x7f76780a1670 con 0x7f7678137940 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): my global_id is 4999 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7660003520 con 0x7f7678137940 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1655575581 0 0) 0x7f76780a2c70 con 0x7f76780a49a0 2024-05-08T15:49:15.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 10 monclient(hunting): my global_id is 94149 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.684+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76780a1670 con 0x7f76780a49a0 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2747003817 0 0) 0x7f7660003520 con 0x7f7678137940 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 >> v1:172.21.15.81:6790/0 conn(0x7f76780a41b0 legacy=0x7f7678140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 >> v1:172.21.15.7:6789/0 conn(0x7f76780a49a0 legacy=0x7f767813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 monclient: found mon.noname-c 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76780a39f0 con 0x7f7678137940 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f76680017b0 con 0x7f7678137940 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 5 monclient: authenticate success, global_id 4999 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: dump: 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.694 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7668001a90 con 0x7f7678137940 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 1 -- 172.21.15.7:0/480378275 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7668004b20 con 0x7f7678137940 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:15.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f7675ffb640 10 monclient: dump: 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.696 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 4 set_mon_vals no callback set 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 10 monclient: shutdown 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 1 -- 172.21.15.7:0/480378275 >> v1:172.21.15.7:6791/0 conn(0x7f7678137940 legacy=0x7f7678137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 1 -- 172.21.15.7:0/480378275 shutdown_connections 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 1 -- 172.21.15.7:0/480378275 >> 172.21.15.7:0/480378275 conn(0x7f76780a9fa0 msgr2=0x7f76780aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 1 -- 172.21.15.7:0/480378275 shutdown_connections 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.685+0000 7f767d5e6640 1 -- 172.21.15.7:0/480378275 wait complete. 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.686+0000 7f767d5e6640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.686+0000 7f767d5e6640 10 monclient: monmap: 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.697 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.698 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.686+0000 7f767d5e6640 1 Processor -- start 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.686+0000 7f767d5e6640 1 -- start start 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.686+0000 7f767d5e6640 10 monclient: init 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 10 monclient: _add_conns ranks=[3,2,4,6,8,5,0,7,1] 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 10 monclient(hunting): picked mon.noname-g con 0x7f76780a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 10 monclient(hunting): picked mon.noname-b con 0x7f76780a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 10 monclient(hunting): picked mon.noname-c con 0x7f7678137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780a1f30 con 0x7f76780a49a0 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780a0f60 con 0x7f76780a41b0 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76781356d0 con 0x7f7678137940 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f767d5e6640 10 monclient(hunting): authenticate will time out at 963.803047s 2024-05-08T15:49:15.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f76777fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f7678137940 0x7f76781d45f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57892/0 (socket says 172.21.15.7:57892) 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f76777fe640 1 -- 172.21.15.7:0/618410638 learned_addr learned my addr 172.21.15.7:0/618410638 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1075660559 0 0) 0x7f76781356d0 con 0x7f7678137940 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): my global_id is 5008 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7664003520 con 0x7f7678137940 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489542216 0 0) 0x7f76780a1f30 con 0x7f76780a49a0 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): my global_id is 94151 2024-05-08T15:49:15.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76781356d0 con 0x7f76780a49a0 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2078040431 0 0) 0x7f76780a0f60 con 0x7f76780a41b0 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 10 monclient(hunting): my global_id is 75081 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76780a1f30 con 0x7f76780a41b0 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 438128141 0 0) 0x7f7664003520 con 0x7f7678137940 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.687+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76780a0f60 con 0x7f7678137940 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7670001de0 con 0x7f7678137940 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): dump: 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.701 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.702 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3207911206 0 0) 0x7f76781356d0 con 0x7f76780a49a0 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7664003520 con 0x7f76780a49a0 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f766c001320 con 0x7f76780a49a0 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): dump: 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.703 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3567692182 0 0) 0x7f76780a1f30 con 0x7f76780a41b0 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76781356d0 con 0x7f76780a41b0 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7668001770 con 0x7f76780a41b0 2024-05-08T15:49:15.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient(hunting): dump: 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.705 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 946008284 0 0) 0x7f76780a0f60 con 0x7f7678137940 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.81:6790/0 conn(0x7f76780a41b0 legacy=0x7f76781cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.7:6790/0 conn(0x7f76780a49a0 legacy=0x7f7678141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 monclient: found mon.c 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7678135f90 con 0x7f7678137940 2024-05-08T15:49:15.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f767d5e6640 5 monclient: authenticate success, global_id 5008 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f767d5e6640 10 monclient: _renew_subs 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f767d5e6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f76781417a0 con 0x7f7678137940 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f767d5e6640 10 monclient: _renew_subs 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f767d5e6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f7678136410 con 0x7f7678137940 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7670003c40 con 0x7f7678137940 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f76700041b0 con 0x7f7678137940 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f76757fa640 4 set_mon_vals no callback set 2024-05-08T15:49:15.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.688+0000 7f765f7fe640 10 monclient: dump: 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.708 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f767d5e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f76580065c0 con 0x7f7658002860 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7670004350 con 0x7f7678137940 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(97..97 src has 1..97) ==== 3898+0+0 (unknown 1278706646 0 0) 0x7f76700471b0 con 0x7f7678137940 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50427466 0 0) 0x7f76580065c0 con 0x7f7658002860 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 10 monclient(hunting): my global_id is 4942 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76780a0f60 con 0x7f7658002860 2024-05-08T15:49:15.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f76767fc640 10 monclient: get_auth_request con 0x7f766403a480 auth_method 0 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1715845687 0 0) 0x7f76780a0f60 con 0x7f7658002860 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f76580065c0 con 0x7f7658002860 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7668004b40 con 0x7f7658002860 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.689+0000 7f765f7fe640 10 monclient: dump: 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.710 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.690+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1162492916 0 0) 0x7f76580065c0 con 0x7f7658002860 2024-05-08T15:49:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.690+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f765800a150 con 0x7f7658002860 2024-05-08T15:49:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.690+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7668004f10 con 0x7f7658002860 2024-05-08T15:49:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.690+0000 7f765f7fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.690+0000 7f765f7fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.690+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.81:6789/0 conn(0x7f7658002860 legacy=0x7f7658004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.690+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.81:6789/0 conn(0x7f7658002860 legacy=0x7f7658004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.696+0000 7f767d5e6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:15.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.696+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7658009e50 con 0x7f7658009110 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2894452140 0 0) 0x7f7658009e50 con 0x7f7658009110 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 10 monclient(hunting): my global_id is 4951 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76580065c0 con 0x7f7658009110 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4163224542 0 0) 0x7f76580065c0 con 0x7f7658009110 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7658009e50 con 0x7f7658009110 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7668005710 con 0x7f7658009110 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 10 monclient: dump: 2024-05-08T15:49:15.714 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.715 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4034594961 0 0) 0x7f7658009e50 con 0x7f7658009110 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.697+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f765800c3d0 con 0x7f7658009110 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7695 (unknown 0 0 3900478756) 0x7f7668005a60 con 0x7f7658009110 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f765f7fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f765f7fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.81:6789/0 conn(0x7f7658009110 legacy=0x7f76580095a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f765f7fe640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.81:6789/0 conn(0x7f7658009110 legacy=0x7f76580095a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.716 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 148014, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:15.717 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.718 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.719 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:15.720 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.721 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.724 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.725 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.726 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.727 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:15.728 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.729 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.730 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.731 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:15.732 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.733 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.734 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.735 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.736 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:15.736 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:15.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.81:6812/847627118 conn(0x7f766403a480 legacy=0x7f766403c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 10 monclient: shutdown 2024-05-08T15:49:15.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 >> v1:172.21.15.7:6791/0 conn(0x7f7678137940 legacy=0x7f76781d45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f76777fe640 1 -- 172.21.15.7:0/618410638 reap_dead start 2024-05-08T15:49:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 shutdown_connections 2024-05-08T15:49:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 >> 172.21.15.7:0/618410638 conn(0x7f76780a9fa0 msgr2=0x7f76781307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 shutdown_connections 2024-05-08T15:49:15.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.698+0000 7f767d5e6640 1 -- 172.21.15.7:0/618410638 wait complete. 2024-05-08T15:49:15.738 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:15.738 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:49:15.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:49:15.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.844+0000 7f00a6ef8640 10 monclient: get_monmap_and_config 2024-05-08T15:49:15.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.844+0000 7f00a6ef8640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.844+0000 7f00a6ef8640 10 monclient: monmap: 2024-05-08T15:49:15.842 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.842 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.842 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:15.846870+0000 2024-05-08T15:49:15.842 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:15.846870+0000 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.843 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.845+0000 7f00a6ef8640 1 Processor -- start 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.845+0000 7f00a6ef8640 1 -- start start 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.845+0000 7f00a6ef8640 10 monclient: init 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 10 monclient: _add_conns ranks=[3,5,1,0,7,2,8,6,4] 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 10 monclient(hunting): picked mon.noname-g con 0x7f00a0137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 10 monclient(hunting): picked mon.noname-h con 0x7f00a00a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 10 monclient(hunting): picked mon.noname-f con 0x7f00a00a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a00a2c70 con 0x7f00a00a41b0 2024-05-08T15:49:15.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a00a2f50 con 0x7f00a0137940 2024-05-08T15:49:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a00a1670 con 0x7f00a00a49a0 2024-05-08T15:49:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a6ef8640 10 monclient(hunting): authenticate will time out at 963.961840s 2024-05-08T15:49:15.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a5495640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f00a00a41b0 0x7f00a0140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:43446/0 (socket says 172.21.15.7:43446) 2024-05-08T15:49:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f00a5495640 1 -- 172.21.15.7:0/3531007508 learned_addr learned my addr 172.21.15.7:0/3531007508 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009ffff640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7f00a00a49a0 0x7f00a013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:36906/0 (socket says 172.21.15.7:36906) 2024-05-08T15:49:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756769950 0 0) 0x7f00a00a2f50 con 0x7f00a0137940 2024-05-08T15:49:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): my global_id is 75090 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f008c003520 con 0x7f00a0137940 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326177732 0 0) 0x7f00a00a1670 con 0x7f00a00a49a0 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): my global_id is 4973 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a00a2f50 con 0x7f00a00a49a0 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 438702100 0 0) 0x7f00a00a2c70 con 0x7f00a00a41b0 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 10 monclient(hunting): my global_id is 4960 2024-05-08T15:49:15.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.846+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a00a1670 con 0x7f00a00a41b0 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2574092271 0 0) 0x7f008c003520 con 0x7f00a0137940 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 >> v1:172.21.15.81:6791/0 conn(0x7f00a00a49a0 legacy=0x7f00a013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 >> v1:172.21.15.81:6789/0 conn(0x7f00a00a41b0 legacy=0x7f00a0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 monclient: found mon.noname-g 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a00a39f0 con 0x7f00a0137940 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0090001a60 con 0x7f00a0137940 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: dump: 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.857 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 5 monclient: authenticate success, global_id 75090 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0090001d80 con 0x7f00a0137940 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 1 -- 172.21.15.7:0/3531007508 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0090003cd0 con 0x7f00a0137940 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 4 set_mon_vals no callback set 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f009f7fe640 10 monclient: dump: 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.859 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 10 monclient: shutdown 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3531007508 >> v1:172.21.15.81:6790/0 conn(0x7f00a0137940 legacy=0x7f00a0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3531007508 shutdown_connections 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3531007508 >> 172.21.15.7:0/3531007508 conn(0x7f00a00a9fa0 msgr2=0x7f00a00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3531007508 shutdown_connections 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.847+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3531007508 wait complete. 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.848+0000 7f00a6ef8640 10 monclient: build_initial_monmap 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.848+0000 7f00a6ef8640 10 monclient: monmap: 2024-05-08T15:49:15.860 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:15.861 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.848+0000 7f00a6ef8640 1 Processor -- start 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.848+0000 7f00a6ef8640 1 -- start start 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.848+0000 7f00a6ef8640 10 monclient: init 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 10 monclient: _add_conns ranks=[5,1,2,3,6,0,7,8,4] 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 10 monclient(hunting): picked mon.noname-h con 0x7f00a00a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 10 monclient(hunting): picked mon.noname-f con 0x7f00a00a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 10 monclient(hunting): picked mon.noname-b con 0x7f00a0137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a01c9ac0 con 0x7f00a00a49a0 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a0135490 con 0x7f00a0137940 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00a0135770 con 0x7f00a00a41b0 2024-05-08T15:49:15.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a6ef8640 10 monclient(hunting): authenticate will time out at 963.964982s 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a5495640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f00a0137940 0x7f00a01d4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43582/0 (socket says 172.21.15.7:43582) 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f00a5495640 1 -- 172.21.15.7:0/3625290631 learned_addr learned my addr 172.21.15.7:0/3625290631 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772446402 0 0) 0x7f00a0135490 con 0x7f00a0137940 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): my global_id is 94160 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0080003520 con 0x7f00a0137940 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4128795928 0 0) 0x7f00a0135770 con 0x7f00a00a41b0 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): my global_id is 4982 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a0135490 con 0x7f00a00a41b0 2024-05-08T15:49:15.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213164415 0 0) 0x7f00a01c9ac0 con 0x7f00a00a49a0 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 10 monclient(hunting): my global_id is 4969 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a0135770 con 0x7f00a00a49a0 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 88627154 0 0) 0x7f0080003520 con 0x7f00a0137940 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.849+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00a01c9ac0 con 0x7f00a0137940 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f00980019d0 con 0x7f00a0137940 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient(hunting): dump: 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.864 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:15.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2661126673 0 0) 0x7f00a0135490 con 0x7f00a00a41b0 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0080003520 con 0x7f00a00a41b0 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0090001a20 con 0x7f00a00a41b0 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient(hunting): dump: 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.866 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1601561065 0 0) 0x7f00a01c9ac0 con 0x7f00a0137940 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.81:6791/0 conn(0x7f00a00a41b0 legacy=0x7f00a01c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.81:6789/0 conn(0x7f00a00a49a0 legacy=0x7f00a0136f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 monclient: found mon.b 2024-05-08T15:49:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a0135a50 con 0x7f00a0137940 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: _finish_auth 0 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f00a6ef8640 5 monclient: authenticate success, global_id 94160 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f00a6ef8640 10 monclient: _renew_subs 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f00a6ef8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f00a0137690 con 0x7f00a0137940 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f00a6ef8640 10 monclient: _renew_subs 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f00a6ef8640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f00a01342c0 con 0x7f00a0137940 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0098001ce0 con 0x7f00a0137940 2024-05-08T15:49:15.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0098004dd0 con 0x7f00a0137940 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009effd640 4 set_mon_vals no callback set 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.850+0000 7f009d7fa640 10 monclient: dump: 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.869 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f0098004f70 con 0x7f00a0137940 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f00a6ef8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(97..97 src has 1..97) ==== 3898+0+0 (unknown 1278706646 0 0) 0x7f0098046df0 con 0x7f00a0137940 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f00840065c0 con 0x7f0084002860 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f00a4c94640 10 monclient: get_auth_request con 0x7f008003a020 auth_method 0 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4037410751 0 0) 0x7f00840065c0 con 0x7f0084002860 2024-05-08T15:49:15.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 10 monclient(hunting): my global_id is 75099 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00a01c9ac0 con 0x7f0084002860 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1301031326 0 0) 0x7f00a01c9ac0 con 0x7f0084002860 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00840065c0 con 0x7f0084002860 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.851+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0088004b20 con 0x7f0084002860 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 10 monclient: dump: 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.871 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3109062128 0 0) 0x7f00840065c0 con 0x7f0084002860 2024-05-08T15:49:15.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f008400a150 con 0x7f0084002860 2024-05-08T15:49:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0088004ef0 con 0x7f0084002860 2024-05-08T15:49:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.81:6790/0 conn(0x7f0084002860 legacy=0x7f0084004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.852+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.81:6790/0 conn(0x7f0084002860 legacy=0x7f0084004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f00a6ef8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0084009e40 con 0x7f0084009100 2024-05-08T15:49:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 634073395 0 0) 0x7f0084009e40 con 0x7f0084009100 2024-05-08T15:49:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f009d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f009d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f009d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:15.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f009d7fa640 10 monclient(hunting): my global_id is 75108 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00840065c0 con 0x7f0084009100 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.859+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 758566747 0 0) 0x7f00840065c0 con 0x7f0084009100 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0084009e40 con 0x7f0084009100 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0088005690 con 0x7f0084009100 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 10 monclient: dump: 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:15.875 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:15.876 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3222467968 0 0) 0x7f0084009e40 con 0x7f0084009100 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f008400c3d0 con 0x7f0084009100 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 1986943169) 0x7f00880069f0 con 0x7f0084009100 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.81:6790/0 conn(0x7f0084009100 legacy=0x7f0084009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f009d7fa640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.81:6790/0 conn(0x7f0084009100 legacy=0x7f0084009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:15.877 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 148151, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:15.878 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.879 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.880 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:15.881 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:15.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:15.883 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.884 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.886 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:15.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.888 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.889 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.890 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:15.891 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.892 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:15.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:15.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:15.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:15.897 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:15.897 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.81:6812/847627118 conn(0x7f008003a020 legacy=0x7f008003c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f00a6ef8640 10 monclient: shutdown 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f00a6ef8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 >> v1:172.21.15.7:6790/0 conn(0x7f00a0137940 legacy=0x7f00a01d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f00a5495640 1 -- 172.21.15.7:0/3625290631 reap_dead start 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.860+0000 7f00a5495640 1 -- 172.21.15.7:0/3625290631 reap_dead start 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.861+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 shutdown_connections 2024-05-08T15:49:15.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.861+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 >> 172.21.15.7:0/3625290631 conn(0x7f00a00a9fa0 msgr2=0x7f00a0130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.861+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 shutdown_connections 2024-05-08T15:49:15.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:15.861+0000 7f00a6ef8640 1 -- 172.21.15.7:0/3625290631 wait complete. 2024-05-08T15:49:15.899 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:15.899 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:49:15.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.006+0000 7f068c663640 10 monclient: get_monmap_and_config 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.006+0000 7f068c663640 10 monclient: build_initial_monmap 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.006+0000 7f068c663640 10 monclient: monmap: 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:16.008620+0000 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:16.008620+0000 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.004 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:16.005 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.007+0000 7f068c663640 1 Processor -- start 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.007+0000 7f068c663640 1 -- start start 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.007+0000 7f068c663640 10 monclient: init 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 10 monclient: _add_conns ranks=[4,5,7,3,0,6,8,1,2] 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 10 monclient(hunting): picked mon.noname-c con 0x7f0684137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 10 monclient(hunting): picked mon.noname-h con 0x7f06840a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 10 monclient(hunting): picked mon.noname-i con 0x7f06840a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06840a2c70 con 0x7f0684137940 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06840a2f50 con 0x7f06840a49a0 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06840a1670 con 0x7f06840a41b0 2024-05-08T15:49:16.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 10 monclient(hunting): authenticate will time out at 964.123927s 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068a3ff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f0684137940 0x7f0684137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57902/0 (socket says 172.21.15.7:57902) 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068a3ff640 1 -- 172.21.15.7:0/2662897254 learned_addr learned my addr 172.21.15.7:0/2662897254 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2858520815 0 0) 0x7f06840a2c70 con 0x7f0684137940 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): my global_id is 5017 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0670003520 con 0x7f0684137940 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4246878077 0 0) 0x7f06840a2f50 con 0x7f06840a49a0 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): my global_id is 4991 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06840a2c70 con 0x7f06840a49a0 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945570022 0 0) 0x7f06840a1670 con 0x7f06840a41b0 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient(hunting): my global_id is 94129 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06840a2f50 con 0x7f06840a41b0 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3806084757 0 0) 0x7f0670003520 con 0x7f0684137940 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 >> v1:172.21.15.81:6792/0 conn(0x7f06840a41b0 legacy=0x7f0684140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 >> v1:172.21.15.81:6791/0 conn(0x7f06840a49a0 legacy=0x7f068413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 monclient: found mon.noname-c 2024-05-08T15:49:16.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06840a39f0 con 0x7f0684137940 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient: _finish_auth 0 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f06800017b0 con 0x7f0684137940 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 5 monclient: authenticate success, global_id 5017 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f068c663640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.008+0000 7f06893fd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 10 monclient: dump: 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.017 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.018 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0680001a10 con 0x7f0684137940 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 1 -- 172.21.15.7:0/2662897254 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0680004be0 con 0x7f0684137940 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 4 set_mon_vals no callback set 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f06893fd640 10 monclient: dump: 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.019 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 10 monclient: shutdown 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 1 -- 172.21.15.7:0/2662897254 >> v1:172.21.15.7:6791/0 conn(0x7f0684137940 legacy=0x7f0684137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 1 -- 172.21.15.7:0/2662897254 shutdown_connections 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 1 -- 172.21.15.7:0/2662897254 >> 172.21.15.7:0/2662897254 conn(0x7f06840a9fa0 msgr2=0x7f06840aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 1 -- 172.21.15.7:0/2662897254 shutdown_connections 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.009+0000 7f068c663640 1 -- 172.21.15.7:0/2662897254 wait complete. 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.010+0000 7f068c663640 10 monclient: build_initial_monmap 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.010+0000 7f068c663640 10 monclient: monmap: 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:16.021 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.010+0000 7f068c663640 1 Processor -- start 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.010+0000 7f068c663640 1 -- start start 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.010+0000 7f068c663640 10 monclient: init 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 10 monclient: _add_conns ranks=[0,5,2,3,1,6,8,7,4] 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 10 monclient(hunting): picked mon.noname-a con 0x7f06840a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 10 monclient(hunting): picked mon.noname-h con 0x7f06840a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 10 monclient(hunting): picked mon.noname-b con 0x7f0684137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:16.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0684136180 con 0x7f06840a41b0 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06840a1f30 con 0x7f0684137940 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0684135a80 con 0x7f06840a49a0 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068c663640 10 monclient(hunting): authenticate will time out at 964.126925s 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068a3ff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f06840a41b0 0x7f0684131f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48838/0 (socket says 172.21.15.7:48838) 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f068a3ff640 1 -- 172.21.15.7:0/2725759538 learned_addr learned my addr 172.21.15.7:0/2725759538 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2859239029 0 0) 0x7f06840a39f0 con 0x7f06840a41b0 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): my global_id is 94158 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0668003520 con 0x7f06840a41b0 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034845130 0 0) 0x7f06840a2f50 con 0x7f0684137940 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): my global_id is 94169 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06840a39f0 con 0x7f0684137940 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3392610903 0 0) 0x7f0684135a80 con 0x7f06840a49a0 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): my global_id is 5000 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06840a2f50 con 0x7f06840a49a0 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2976083620 0 0) 0x7f06840a39f0 con 0x7f0684137940 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0684135a80 con 0x7f0684137940 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f067c001c80 con 0x7f0684137940 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:16.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): dump: 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.025 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 514458174 0 0) 0x7f0668003520 con 0x7f06840a41b0 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f06840a39f0 con 0x7f06840a41b0 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 700947297 0 0) 0x7f06840a2f50 con 0x7f06840a49a0 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.011+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0668003520 con 0x7f06840a49a0 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0680004b20 con 0x7f06840a41b0 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient(hunting): dump: 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.026 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.027 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0674001a10 con 0x7f06840a49a0 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient(hunting): dump: 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.028 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 874199837 0 0) 0x7f0684135a80 con 0x7f0684137940 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.81:6791/0 conn(0x7f06840a49a0 legacy=0x7f0684132630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.7:6789/0 conn(0x7f06840a41b0 legacy=0x7f0684131f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 monclient: found mon.b 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:16.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06840a0f60 con 0x7f0684137940 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: _finish_auth 0 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 5 monclient: authenticate success, global_id 94169 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 10 monclient: _renew_subs 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0684135000 con 0x7f0684137940 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 10 monclient: _renew_subs 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f06841357a0 con 0x7f0684137940 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f067c0020b0 con 0x7f0684137940 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:16.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f067c003f80 con 0x7f0684137940 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f0688bfc640 4 set_mon_vals no callback set 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f067affd640 10 monclient: dump: 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.031 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.012+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f066c0065c0 con 0x7f066c002860 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f067c0041a0 con 0x7f0684137940 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(97..97 src has 1..97) ==== 3898+0+0 (unknown 1278706646 0 0) 0x7f067c046050 con 0x7f0684137940 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244677108 0 0) 0x7f066c0065c0 con 0x7f066c002860 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 10 monclient(hunting): my global_id is 5009 2024-05-08T15:49:16.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0684135a80 con 0x7f066c002860 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f0689bfe640 10 monclient: get_auth_request con 0x7f066803a4d0 auth_method 0 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3255877992 0 0) 0x7f0684135a80 con 0x7f066c002860 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f066c0065c0 con 0x7f066c002860 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0680004d50 con 0x7f066c002860 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 10 monclient: dump: 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.033 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1588130604 0 0) 0x7f066c0065c0 con 0x7f066c002860 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.013+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f066c00a150 con 0x7f066c002860 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.014+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0680007480 con 0x7f066c002860 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.014+0000 7f067affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.014+0000 7f067affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.014+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.81:6791/0 conn(0x7f066c002860 legacy=0x7f066c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.014+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.81:6791/0 conn(0x7f066c002860 legacy=0x7f066c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.020+0000 7f068c663640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.020+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f066c009e40 con 0x7f066c009100 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3886586380 0 0) 0x7f066c009e40 con 0x7f066c009100 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 10 monclient(hunting): my global_id is 5018 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f066c0065c0 con 0x7f066c009100 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1990832172 0 0) 0x7f066c0065c0 con 0x7f066c009100 2024-05-08T15:49:16.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f066c009e40 con 0x7f066c009100 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0680005630 con 0x7f066c009100 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 10 monclient: dump: 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.037 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 356988718 0 0) 0x7f066c009e40 con 0x7f066c009100 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.021+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f066c00c3d0 con 0x7f066c009100 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 861367236) 0x7f0680006b10 con 0x7f066c009100 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f067affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f067affd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.81:6791/0 conn(0x7f066c009100 legacy=0x7f066c009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f067affd640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.81:6791/0 conn(0x7f066c009100 legacy=0x7f066c009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 148329, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:16.039 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:16.040 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:16.041 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:16.042 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:16.043 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:16.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.046 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:16.047 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.049 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:16.050 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:16.052 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.053 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.055 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:16.056 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.81:6812/847627118 conn(0x7f066803a4d0 legacy=0x7f066803c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 10 monclient: shutdown 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 >> v1:172.21.15.7:6790/0 conn(0x7f0684137940 legacy=0x7f06841d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068ac00640 1 -- 172.21.15.7:0/2725759538 reap_dead start 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068ac00640 1 -- 172.21.15.7:0/2725759538 reap_dead start 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 shutdown_connections 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 >> 172.21.15.7:0/2725759538 conn(0x7f06840a9fa0 msgr2=0x7f0684130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 shutdown_connections 2024-05-08T15:49:16.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.022+0000 7f068c663640 1 -- 172.21.15.7:0/2725759538 wait complete. 2024-05-08T15:49:16.059 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:16.059 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:49:16.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.166+0000 7f62688d2640 10 monclient: get_monmap_and_config 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.166+0000 7f62688d2640 10 monclient: build_initial_monmap 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.166+0000 7f62688d2640 10 monclient: monmap: 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:16.167900+0000 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:16.167900+0000 2024-05-08T15:49:16.163 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:16.164 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:16.165 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.166+0000 7f62688d2640 1 Processor -- start 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.166+0000 7f62688d2640 1 -- start start 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.166+0000 7f62688d2640 10 monclient: init 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient: _add_conns ranks=[3,7,6,5,4,1,0,8,2] 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient(hunting): picked mon.noname-g con 0x7f6264137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient(hunting): picked mon.noname-i con 0x7f6264138430 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient(hunting): picked mon.noname-d con 0x7f62640a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:16.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62640a2990 con 0x7f6264137940 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62640a2c70 con 0x7f62640a49a0 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62640a39f0 con 0x7f6264138430 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient(hunting): authenticate will time out at 964.282878s 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f6262d9d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f62640a49a0 0x7f6264140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35520/0 (socket says 172.21.15.7:35520) 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f6262d9d640 1 -- 172.21.15.7:0/1154209517 learned_addr learned my addr 172.21.15.7:0/1154209517 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 299888524 0 0) 0x7f62640a2c70 con 0x7f62640a49a0 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient(hunting): my global_id is 75030 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f624c003520 con 0x7f62640a49a0 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3347092614 0 0) 0x7f624c003520 con 0x7f62640a49a0 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 >> v1:172.21.15.81:6792/0 conn(0x7f6264138430 legacy=0x7f626413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 >> v1:172.21.15.81:6790/0 conn(0x7f6264137940 legacy=0x7f6264137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: _finish_hunting 0 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 monclient: found mon.noname-d 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62640a4290 con 0x7f62640a49a0 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: _finish_auth 0 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: _check_auth_tickets 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f625c001b30 con 0x7f62640a49a0 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: dump: 2024-05-08T15:49:16.176 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.177 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 5 monclient: authenticate success, global_id 75030 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f625c001e20 con 0x7f62640a49a0 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 1 -- 172.21.15.7:0/1154209517 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f625c004f60 con 0x7f62640a49a0 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f626159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.167+0000 7f62688d2640 4 set_mon_vals no callback set 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f626159a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f626159a640 10 monclient: dump: 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.179 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f62688d2640 10 monclient: shutdown 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f62688d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f62688d2640 1 -- 172.21.15.7:0/1154209517 >> v1:172.21.15.7:6792/0 conn(0x7f62640a49a0 legacy=0x7f6264140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f62688d2640 1 -- 172.21.15.7:0/1154209517 shutdown_connections 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f62688d2640 1 -- 172.21.15.7:0/1154209517 >> 172.21.15.7:0/1154209517 conn(0x7f62640a9fa0 msgr2=0x7f62640aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f62688d2640 1 -- 172.21.15.7:0/1154209517 shutdown_connections 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.168+0000 7f62688d2640 1 -- 172.21.15.7:0/1154209517 wait complete. 2024-05-08T15:49:16.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.169+0000 7f62688d2640 10 monclient: build_initial_monmap 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.169+0000 7f62688d2640 10 monclient: monmap: 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:16.181 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.169+0000 7f62688d2640 1 Processor -- start 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.169+0000 7f62688d2640 1 -- start start 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.169+0000 7f62688d2640 10 monclient: init 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.169+0000 7f62688d2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 10 monclient: _add_conns ranks=[6,3,7,2,1,8,4,0,5] 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 10 monclient(hunting): picked mon.noname-d con 0x7f62640a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:16.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 10 monclient(hunting): picked mon.noname-g con 0x7f6264137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 10 monclient(hunting): picked mon.noname-i con 0x7f6264138430 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62640a0f60 con 0x7f6264137940 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6264135760 con 0x7f62640a49a0 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6264135a40 con 0x7f6264138430 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f62688d2640 10 monclient(hunting): authenticate will time out at 964.285729s 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f626259c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f62640a49a0 0x7f6264131e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35536/0 (socket says 172.21.15.7:35536) 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f626259c640 1 -- 172.21.15.7:0/1420990226 learned_addr learned my addr 172.21.15.7:0/1420990226 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2615183072 0 0) 0x7f6264135760 con 0x7f62640a49a0 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): my global_id is 75039 2024-05-08T15:49:16.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6244003520 con 0x7f62640a49a0 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1410622495 0 0) 0x7f62640a39f0 con 0x7f6264137940 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): my global_id is 75126 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6264135760 con 0x7f6264137940 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3402124931 0 0) 0x7f6264135a40 con 0x7f6264138430 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): my global_id is 94147 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62640a39f0 con 0x7f6264138430 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 679900771 0 0) 0x7f6244003520 con 0x7f62640a49a0 2024-05-08T15:49:16.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6264135a40 con 0x7f62640a49a0 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6254001b10 con 0x7f62640a49a0 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.185 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 88934944 0 0) 0x7f62640a39f0 con 0x7f6264138430 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6244003520 con 0x7f6264138430 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1496083270 0 0) 0x7f6264135760 con 0x7f6264137940 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f62640a39f0 con 0x7f6264137940 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f625c004ea0 con 0x7f6264138430 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:16.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.187 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6258001760 con 0x7f6264137940 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.188 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 392386997 0 0) 0x7f6264135a40 con 0x7f62640a49a0 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.81:6792/0 conn(0x7f6264138430 legacy=0x7f62641d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.170+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.81:6790/0 conn(0x7f6264137940 legacy=0x7f6264132570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 1 monclient: found mon.d 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:16.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62640a1f30 con 0x7f62640a49a0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 5 monclient: authenticate success, global_id 75039 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 10 monclient: _renew_subs 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f62641339c0 con 0x7f62640a49a0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 10 monclient: _renew_subs 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f6264135d20 con 0x7f62640a49a0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f62540038c0 con 0x7f62640a49a0 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:16.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6254003db0 con 0x7f62640a49a0 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6260d99640 4 set_mon_vals no callback set 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 10 monclient: dump: 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.191 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f62480065c0 con 0x7f6248002860 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.171+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6254003fd0 con 0x7f62640a49a0 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(97..97 src has 1..97) ==== 3898+0+0 (unknown 1278706646 0 0) 0x7f6254046f70 con 0x7f62640a49a0 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3627303601 0 0) 0x7f62480065c0 con 0x7f6248002860 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 10 monclient(hunting): my global_id is 94156 2024-05-08T15:49:16.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6264135a40 con 0x7f6248002860 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6262d9d640 10 monclient: get_auth_request con 0x7f624403a4d0 auth_method 0 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3257928884 0 0) 0x7f6264135a40 con 0x7f6248002860 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f62480065c0 con 0x7f6248002860 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6258003950 con 0x7f6248002860 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 10 monclient: dump: 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 362364911 0 0) 0x7f62480065c0 con 0x7f6248002860 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.172+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f624800a150 con 0x7f6248002860 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.173+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f6258004640 con 0x7f6248002860 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.173+0000 7f6252ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.173+0000 7f6252ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:16.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.173+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.81:6792/0 conn(0x7f6248002860 legacy=0x7f6248004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.173+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.81:6792/0 conn(0x7f6248002860 legacy=0x7f6248004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:16.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f62688d2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:16.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6248009e20 con 0x7f62480090e0 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3621158491 0 0) 0x7f6248009e20 con 0x7f62480090e0 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 10 monclient(hunting): my global_id is 94165 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62480065c0 con 0x7f62480090e0 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3525682659 0 0) 0x7f62480065c0 con 0x7f62480090e0 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6248009e20 con 0x7f62480090e0 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6258004160 con 0x7f62480090e0 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 10 monclient: dump: 2024-05-08T15:49:16.198 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.199 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2609283841 0 0) 0x7f6248009e20 con 0x7f62480090e0 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.180+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f624800c390 con 0x7f62480090e0 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 3934782063) 0x7f62580043c0 con 0x7f62480090e0 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f6252ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f6252ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.81:6792/0 conn(0x7f62480090e0 legacy=0x7f6248009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f6252ffd640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.81:6792/0 conn(0x7f62480090e0 legacy=0x7f6248009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:16.200 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 6744, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.201 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:16.202 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:16.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:16.204 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:16.205 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:16.207 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:16.209 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.212 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:16.213 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.216 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:16.218 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:16.219 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.81:6812/847627118 conn(0x7f624403a4d0 legacy=0x7f624403c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 10 monclient: shutdown 2024-05-08T15:49:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 >> v1:172.21.15.7:6792/0 conn(0x7f62640a49a0 legacy=0x7f6264131e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f6262d9d640 1 -- 172.21.15.7:0/1420990226 reap_dead start 2024-05-08T15:49:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f6262d9d640 1 -- 172.21.15.7:0/1420990226 reap_dead start 2024-05-08T15:49:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 shutdown_connections 2024-05-08T15:49:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 >> 172.21.15.7:0/1420990226 conn(0x7f62640a9fa0 msgr2=0x7f62641308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 shutdown_connections 2024-05-08T15:49:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.181+0000 7f62688d2640 1 -- 172.21.15.7:0/1420990226 wait complete. 2024-05-08T15:49:16.222 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:16.222 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T15:49:16.222 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T15:49:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.327+0000 7f0075ae6640 10 monclient: get_monmap_and_config 2024-05-08T15:49:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.327+0000 7f0075ae6640 10 monclient: build_initial_monmap 2024-05-08T15:49:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.327+0000 7f0075ae6640 10 monclient: monmap: 2024-05-08T15:49:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:16.329922+0000 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:16.329922+0000 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:16.327 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.328+0000 7f0075ae6640 1 Processor -- start 2024-05-08T15:49:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.328+0000 7f0075ae6640 1 -- start start 2024-05-08T15:49:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.328+0000 7f0075ae6640 10 monclient: init 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 10 monclient: _add_conns ranks=[6,4,8,3,0,2,1,5,7] 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 10 monclient(hunting): picked mon.noname-d con 0x7f0070137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 10 monclient(hunting): picked mon.noname-c con 0x7f00700a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 10 monclient(hunting): picked mon.noname-e con 0x7f00700a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700a2c70 con 0x7f00700a49a0 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700a2f50 con 0x7f0070137940 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700a1670 con 0x7f00700a41b0 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 10 monclient(hunting): authenticate will time out at 964.444856s 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006effd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f00700a49a0 0x7f007013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57908/0 (socket says 172.21.15.7:57908) 2024-05-08T15:49:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006f7fe640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f0070137940 0x7f0070137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35538/0 (socket says 172.21.15.7:35538) 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006ffff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f00700a41b0 0x7f0070140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:57810/0 (socket says 172.21.15.7:57810) 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006effd640 1 -- 172.21.15.7:0/920709834 learned_addr learned my addr 172.21.15.7:0/920709834 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 369963971 0 0) 0x7f00700a2f50 con 0x7f0070137940 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): my global_id is 75048 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0058003520 con 0x7f0070137940 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560451396 0 0) 0x7f00700a1670 con 0x7f00700a41b0 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): my global_id is 94148 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00700a2f50 con 0x7f00700a41b0 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025640036 0 0) 0x7f00700a2c70 con 0x7f00700a49a0 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient(hunting): my global_id is 5026 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00700a1670 con 0x7f00700a49a0 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 152970023 0 0) 0x7f0058003520 con 0x7f0070137940 2024-05-08T15:49:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 >> v1:172.21.15.7:6793/0 conn(0x7f00700a41b0 legacy=0x7f0070140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 >> v1:172.21.15.7:6791/0 conn(0x7f00700a49a0 legacy=0x7f007013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 monclient: found mon.noname-d 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00700a39f0 con 0x7f0070137940 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient: _finish_auth 0 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 5 monclient: authenticate success, global_id 75048 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f00600017b0 con 0x7f0070137940 2024-05-08T15:49:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f006e7fc640 10 monclient: dump: 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.329+0000 7f0075ae6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0060001ad0 con 0x7f0070137940 2024-05-08T15:49:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f006e7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f006e7fc640 1 -- 172.21.15.7:0/920709834 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0060001d40 con 0x7f0070137940 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f006e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 4 set_mon_vals no callback set 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f006e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f006e7fc640 10 monclient: dump: 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 10 monclient: shutdown 2024-05-08T15:49:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 1 -- 172.21.15.7:0/920709834 >> v1:172.21.15.7:6792/0 conn(0x7f0070137940 legacy=0x7f0070137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 1 -- 172.21.15.7:0/920709834 shutdown_connections 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 1 -- 172.21.15.7:0/920709834 >> 172.21.15.7:0/920709834 conn(0x7f00700a9fa0 msgr2=0x7f00700aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 1 -- 172.21.15.7:0/920709834 shutdown_connections 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.330+0000 7f0075ae6640 1 -- 172.21.15.7:0/920709834 wait complete. 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.331+0000 7f0075ae6640 10 monclient: build_initial_monmap 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.331+0000 7f0075ae6640 10 monclient: monmap: 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.331+0000 7f0075ae6640 1 Processor -- start 2024-05-08T15:49:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.331+0000 7f0075ae6640 1 -- start start 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.331+0000 7f0075ae6640 10 monclient: init 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 10 monclient: _add_conns ranks=[5,4,2,6,0,7,3,8,1] 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 10 monclient(hunting): picked mon.noname-h con 0x7f00700a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 10 monclient(hunting): picked mon.noname-c con 0x7f00700a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 10 monclient(hunting): picked mon.noname-b con 0x7f0070137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0070132b80 con 0x7f0070137940 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700a1f30 con 0x7f00700a49a0 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00700a0f60 con 0x7f00700a41b0 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0075ae6640 10 monclient(hunting): authenticate will time out at 964.447829s 2024-05-08T15:49:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f006effd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f00700a49a0 0x7f0070132470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57918/0 (socket says 172.21.15.7:57918) 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f006effd640 1 -- 172.21.15.7:0/1560980854 learned_addr learned my addr 172.21.15.7:0/1560980854 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829358702 0 0) 0x7f00700a1f30 con 0x7f00700a49a0 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): my global_id is 5035 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f005c003520 con 0x7f00700a49a0 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1158551685 0 0) 0x7f0070132b80 con 0x7f0070137940 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): my global_id is 94178 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00700a1f30 con 0x7f0070137940 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 267487308 0 0) 0x7f00700a0f60 con 0x7f00700a41b0 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): my global_id is 5027 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0070132b80 con 0x7f00700a41b0 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1964894678 0 0) 0x7f005c003520 con 0x7f00700a49a0 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00700a0f60 con 0x7f00700a49a0 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0064001a50 con 0x7f00700a49a0 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): dump: 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 927548486 0 0) 0x7f00700a1f30 con 0x7f0070137940 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f005c003520 con 0x7f0070137940 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f00680017b0 con 0x7f0070137940 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 10 monclient(hunting): dump: 2024-05-08T15:49:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1191622625 0 0) 0x7f0070132b80 con 0x7f00700a41b0 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f00700a1f30 con 0x7f00700a41b0 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.332+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 334608316 0 0) 0x7f00700a0f60 con 0x7f00700a49a0 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 >> v1:172.21.15.81:6791/0 conn(0x7f00700a41b0 legacy=0x7f0070131d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 >> v1:172.21.15.7:6790/0 conn(0x7f0070137940 legacy=0x7f00701d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: _finish_hunting 0 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 1 monclient: found mon.c 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0070137640 con 0x7f00700a49a0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: _finish_auth 0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: _check_auth_tickets 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0075ae6640 5 monclient: authenticate success, global_id 5035 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0075ae6640 10 monclient: _renew_subs 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0075ae6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0070134db0 con 0x7f00700a49a0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0075ae6640 10 monclient: _renew_subs 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0075ae6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f00701358a0 con 0x7f00700a49a0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0064004b10 con 0x7f00700a49a0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0064005000 con 0x7f00700a49a0 2024-05-08T15:49:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f006dffb640 4 set_mon_vals no callback set 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.333+0000 7f0057fff640 10 monclient: dump: 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.334+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f0064010380 con 0x7f00700a49a0 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.334+0000 7f0075ae6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.334+0000 7f0075ae6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.334+0000 7f0075ae6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.334+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0050005080 con 0x7f00700a49a0 2024-05-08T15:49:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.334+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(97..97 src has 1..97) ==== 3898+0+0 (unknown 1278706646 0 0) 0x7f0064046e90 con 0x7f00700a49a0 2024-05-08T15:49:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.334+0000 7f006f7fe640 10 monclient: get_auth_request con 0x7f005c03a450 auth_method 0 2024-05-08T15:49:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.339+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f0064015f80 con 0x7f00700a49a0 2024-05-08T15:49:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.339+0000 7f0057fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.339+0000 7f0057fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:16.604 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35309]: starting. 2024-05-08T15:49:16.604 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35309]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35308 2024-05-08T15:49:16.605 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35309]: created object 0... 2024-05-08T15:49:16.605 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35309]: created object 25... 2024-05-08T15:49:16.605 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35309]: created object 49... 2024-05-08T15:49:16.605 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35309]: finishing. 2024-05-08T15:49:16.605 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35309]: shutting down. 2024-05-08T15:49:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.610+0000 7f0075ae6640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-05-08T15:49:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.610+0000 7f0075ae6640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-05-08T15:49:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.610+0000 7f0075ae6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.610+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0050005b80 con 0x7f00700a49a0 2024-05-08T15:49:16.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.611+0000 7f0057fff640 1 -- 172.21.15.7:0/1560980854 <== mon.4 v1:172.21.15.7:6791/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f0064015880 con 0x7f00700a49a0 2024-05-08T15:49:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.612+0000 7f0057fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-05-08T15:49:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.612+0000 7f0057fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 >> v1:172.21.15.81:6812/847627118 conn(0x7f005c03a450 legacy=0x7f005c03c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 10 monclient: shutdown 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 >> v1:172.21.15.7:6791/0 conn(0x7f00700a49a0 legacy=0x7f0070132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 shutdown_connections 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 >> 172.21.15.7:0/1560980854 conn(0x7f00700a9fa0 msgr2=0x7f0070130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 shutdown_connections 2024-05-08T15:49:16.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:16.614+0000 7f0075ae6640 1 -- 172.21.15.7:0/1560980854 wait complete. 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=16,config=2,health=11,kv=10,logm=61} crc {auth=4040911980,config=1618278320,health=3926953126,kv=3560234957,logm=1206171968}) 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=15,config=2,health=12,kv=9,logm=62} crc {auth=2792880774,config=3625383534,health=3977944326,kv=519216810,logm=4070008746}) 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.627 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=1080067076,config=1618278320,health=3977944326,kv=3560234957,logm=427368330}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=16,config=2,health=12,kv=10,logm=60} crc {auth=2685419564,config=1618278320,health=3977944326,kv=3560234957,logm=3713123537}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=16,config=2,health=12,kv=9,logm=61} crc {auth=3534223748,config=1618278320,health=3977944326,kv=368437079,logm=3344530858}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=17,config=2,health=12,kv=10,logm=59} crc {auth=157088035,config=1618278320,health=3977944326,kv=3560234957,logm=400140392}) 2024-05-08T15:49:16.628 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=15,config=2,health=11,kv=10,logm=62} crc {auth=3518328294,config=1618278320,health=3926953126,kv=3560234957,logm=1522676377}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=16,config=1,health=12,kv=10,logm=61} crc {auth=2107783722,config=1228700967,health=1326388786,kv=3560234957,logm=1882499639}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.613+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=17,config=2,health=12,kv=9,logm=60} crc {auth=1080067076,config=1618278320,health=3977944326,kv=67600927,logm=4013390181}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1277946932}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2978581633}) 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.629 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4230473580}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2999467793}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4484734}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1230884784}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4230095516}) 2024-05-08T15:49:16.630 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3416340272}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.614+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3834683099}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2705134946}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1524053745}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=650555633}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3007658603}) 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.631 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3609882001}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1334512039}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3728493023}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=914005747}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.615+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3357653045}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.616+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1138607745}) 2024-05-08T15:49:16.632 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.633 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.633 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2982022639}) 2024-05-08T15:49:16.633 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.633 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.633 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2844667070}) 2024-05-08T15:49:16.633 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.633 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=462117711}) 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3408644810}) 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3658703734}) 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.634 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1294872388}) 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=511730617}) 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.617+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=658455928}) 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3580678504}) 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.635 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3222104973}) 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2226146517}) 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4234054271}) 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.636 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3065289967}) 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4043518465}) 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3569557984}) 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1563931608}) 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.618+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3063124673}) 2024-05-08T15:49:16.637 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=547581451}) 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1496607176}) 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2672594189}) 2024-05-08T15:49:16.638 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4243991934}) 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2316320706}) 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4039693143}) 2024-05-08T15:49:16.639 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2099545892}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4046062825}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.619+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=889103748}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=612740754}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2644849875}) 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.640 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1548662751}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4081279073}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2217769570}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2433913973}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2049277037}) 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.641 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2884543119}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.620+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3521877950}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=34,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=27} crc {logm=1844545855,mdsmap=2840952586,mgr=3460537546,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=440257699,mon_config_key=2097864577,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1133186369}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=11,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=25} crc {logm=2645639453,mdsmap=2840952586,mgr=584111586,mgr_metadata=1815193447,mgrstat=327520701,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=2059246588}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=33,mdsmap=3,mgr=8,mgr_command_descs=1,mgrstat=11,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=4110189187,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgrstat=2675553698,mon_config_key=619004784,monmap=4249362751,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=3690929730}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.642 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=11,mon_config_key=7,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=25} crc {logm=528236131,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=3617009089,mon_config_key=2368181095,monmap=256126146,osd_metadata=2963935437,osd_pg_creating=708651303,osdmap=280093438}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=35,mdsmap=3,mgr=7,mgr_command_descs=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=24} crc {logm=528236131,mdsmap=2840952586,mgr=2196687186,mgr_command_descs=3353810608,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=3347214864}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=35,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=7,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=24} crc {logm=528236131,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=2368181095,monmap=256126145,osd_metadata=2963935437,osd_pg_creating=708651303,osdmap=2901742871}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=35,mdsmap=2,mgr=8,mgr_metadata=1,mgrstat=11,mon_config_key=6,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=27} crc {logm=528236131,mdsmap=3140514009,mgr=584111586,mgr_metadata=1815193447,mgrstat=2427799697,mon_config_key=1068556646,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=2831633313}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=31,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3136133004,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=708651303,osdmap=1405686656}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.622+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=34,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=12,mon_config_key=8,monmap=3,osd_metadata=6,osdmap=24} crc {logm=1535418842,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2465658380,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osdmap=3096589160}) 2024-05-08T15:49:16.643 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=191105767}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=557672037}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2067325167}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1947504898}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3483225223}) 2024-05-08T15:49:16.644 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3308354459}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=583795674}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1078470206}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.623+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=771955291}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=30} crc {osdmap=3308691725}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=27} crc {osdmap=4144547546}) 2024-05-08T15:49:16.645 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=28} crc {osdmap=931365163}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=27} crc {osdmap=3945223384}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=28} crc {osdmap=4019233871}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=29} crc {osdmap=3213281225}) 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.646 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.624+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.647 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.625+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=29} crc {osdmap=1513381222}) 2024-05-08T15:49:16.647 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.625+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:49:16.647 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.625+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=27} crc {osdmap=3023794468}) 2024-05-08T15:49:16.647 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:49:16.625+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=29} crc {osdmap=2478763515}) 2024-05-08T15:49:16.648 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-05-08T15:49:17.546 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35310]: starting. 2024-05-08T15:49:17.546 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35310]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35308 2024-05-08T15:49:17.546 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35310]: shutting down. 2024-05-08T15:49:17.649 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:49:17.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:49:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.771+0000 7f01b231c640 10 monclient: get_monmap_and_config 2024-05-08T15:49:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.771+0000 7f01b231c640 10 monclient: build_initial_monmap 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.771+0000 7f01b231c640 10 monclient: monmap: 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:17.772433+0000 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:17.772433+0000 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:17.768 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:17.769 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:17.769 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 1 Processor -- start 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 1 -- start start 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient: init 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient: _add_conns ranks=[3,1,4,0,5,7,2,8,6] 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient(hunting): picked mon.noname-g con 0x7f01ac137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient(hunting): picked mon.noname-f con 0x7f01ac0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient(hunting): picked mon.noname-c con 0x7f01ac0a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0a2c70 con 0x7f01ac0a49a0 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0a2f50 con 0x7f01ac137940 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0a1670 con 0x7f01ac0a41b0 2024-05-08T15:49:17.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.772+0000 7f01b231c640 10 monclient(hunting): authenticate will time out at 965.887593s 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01b08b9640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f01ac0a41b0 0x7f01ac140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57928/0 (socket says 172.21.15.7:57928) 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01b08b9640 1 -- 172.21.15.7:0/2789046483 learned_addr learned my addr 172.21.15.7:0/2789046483 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 324608045 0 0) 0x7f01ac0a1670 con 0x7f01ac0a41b0 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): my global_id is 5053 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0194003520 con 0x7f01ac0a41b0 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574680823 0 0) 0x7f01ac0a2f50 con 0x7f01ac137940 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:17.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): my global_id is 75144 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01ac0a1670 con 0x7f01ac137940 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3032528015 0 0) 0x7f01ac0a2c70 con 0x7f01ac0a49a0 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient(hunting): my global_id is 4978 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01ac0a2f50 con 0x7f01ac0a49a0 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1164865750 0 0) 0x7f0194003520 con 0x7f01ac0a41b0 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 >> v1:172.21.15.81:6790/0 conn(0x7f01ac137940 legacy=0x7f01ac137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 >> v1:172.21.15.81:6789/0 conn(0x7f01ac0a49a0 legacy=0x7f01ac13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 monclient: found mon.noname-c 2024-05-08T15:49:17.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac0a39f0 con 0x7f01ac0a41b0 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f01a4001b30 con 0x7f01ac0a41b0 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01b231c640 5 monclient: authenticate success, global_id 5053 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01b231c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: dump: 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:17.782 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01b231c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f01a4001e50 con 0x7f01ac0a41b0 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 1 -- 172.21.15.7:0/2789046483 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f01a4004f60 con 0x7f01ac0a41b0 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01aaffd640 10 monclient: dump: 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:17.784 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01b231c640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.773+0000 7f01b231c640 4 set_mon_vals no callback set 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.774+0000 7f01b231c640 10 monclient: shutdown 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.774+0000 7f01b231c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.774+0000 7f01b231c640 1 -- 172.21.15.7:0/2789046483 >> v1:172.21.15.7:6791/0 conn(0x7f01ac0a41b0 legacy=0x7f01ac140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:17.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.774+0000 7f01b231c640 1 -- 172.21.15.7:0/2789046483 shutdown_connections 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.774+0000 7f01b231c640 1 -- 172.21.15.7:0/2789046483 >> 172.21.15.7:0/2789046483 conn(0x7f01ac0a9fa0 msgr2=0x7f01ac0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.774+0000 7f01b231c640 1 -- 172.21.15.7:0/2789046483 shutdown_connections 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.774+0000 7f01b231c640 1 -- 172.21.15.7:0/2789046483 wait complete. 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.775+0000 7f01b231c640 10 monclient: build_initial_monmap 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.775+0000 7f01b231c640 10 monclient: monmap: 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:17.786 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.775+0000 7f01b231c640 1 Processor -- start 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.775+0000 7f01b231c640 1 -- start start 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.775+0000 7f01b231c640 10 monclient: init 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 10 monclient: _add_conns ranks=[8,4,7,5,6,0,2,3,1] 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 10 monclient(hunting): picked mon.noname-e con 0x7f01ac0a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:17.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 10 monclient(hunting): picked mon.noname-c con 0x7f01ac0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 10 monclient(hunting): picked mon.noname-i con 0x7f01ac137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac135660 con 0x7f01ac0a49a0 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac1cc720 con 0x7f01ac137940 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01ac0a1f30 con 0x7f01ac0a41b0 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01b231c640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01ab7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f01ac0a49a0 0x7f01ac1d1390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57936/0 (socket says 172.21.15.7:57936) 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01ab7fe640 1 -- 172.21.15.7:0/1654696348 learned_addr learned my addr 172.21.15.7:0/1654696348 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.776+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813159086 0 0) 0x7f01ac135660 con 0x7f01ac0a49a0 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.777+0000 7f01b231c640 10 monclient(hunting): authenticate will time out at 965.891791s 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.777+0000 7f01a8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.777+0000 7f01a8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.777+0000 7f01a8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.777+0000 7f01a8ff9640 10 monclient(hunting): my global_id is 5062 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.777+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0198003520 con 0x7f01ac0a49a0 2024-05-08T15:49:17.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077853426 0 0) 0x7f01ac0a39f0 con 0x7f01ac137940 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): my global_id is 94210 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01ac135660 con 0x7f01ac137940 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 614574524 0 0) 0x7f01ac0a1f30 con 0x7f01ac0a41b0 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): my global_id is 94193 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f01ac0a39f0 con 0x7f01ac0a41b0 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1874904074 0 0) 0x7f0198003520 con 0x7f01ac0a49a0 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f01ac0a1f30 con 0x7f01ac0a49a0 2024-05-08T15:49:17.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f01a0001b50 con 0x7f01ac0a49a0 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:17.790 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4171438078 0 0) 0x7f01ac135660 con 0x7f01ac137940 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0198003520 con 0x7f01ac137940 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.779+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1063454227 0 0) 0x7f01ac0a39f0 con 0x7f01ac0a41b0 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0198008e60 con 0x7f01ac0a41b0 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f019c001970 con 0x7f01ac0a41b0 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:17.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:17.792 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f01a4004ea0 con 0x7f01ac137940 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:17.793 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2763282719 0 0) 0x7f0198003520 con 0x7f01ac137940 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 >> v1:172.21.15.7:6793/0 conn(0x7f01ac0a41b0 legacy=0x7f01ac141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 >> v1:172.21.15.7:6791/0 conn(0x7f01ac0a49a0 legacy=0x7f01ac1d1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:17.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 monclient: found mon.i 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac0a0f60 con 0x7f01ac137940 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: _finish_auth 0 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f01a40013c0 con 0x7f01ac137940 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f01a4004ea0 con 0x7f01ac137940 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01a8ff9640 10 monclient: dump: 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:17.795 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:17.796 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.780+0000 7f01aa7fc640 4 set_mon_vals no callback set 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.782+0000 7f01b231c640 5 monclient: authenticate success, global_id 94210 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.782+0000 7f01b231c640 10 monclient: _renew_subs 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.782+0000 7f01b231c640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.782+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac137180 con 0x7f01ac137940 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.782+0000 7f01b231c640 10 monclient: _renew_subs 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.782+0000 7f01b231c640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:49:17.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.782+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f01ac1363a0 con 0x7f01ac137940 2024-05-08T15:49:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.784+0000 7f01b231c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.784+0000 7f01b231c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.784+0000 7f01b231c640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:49:17.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.784+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ac0753d0 con 0x7f01ac137940 2024-05-08T15:49:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.784+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f01a4001e50 con 0x7f01ac137940 2024-05-08T15:49:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.784+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(99..99 src has 1..99) ==== 3506+0+0 (unknown 992023057 0 0) 0x7f01a4046920 con 0x7f01ac137940 2024-05-08T15:49:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.785+0000 7f01abfff640 10 monclient: get_auth_request con 0x7f019803aa30 auth_method 0 2024-05-08T15:49:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.789+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f01ac137180 con 0x7f01ac137940 2024-05-08T15:49:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.789+0000 7f01a8ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:17.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:17.789+0000 7f01a8ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.044+0000 7f01b231c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:49:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.044+0000 7f01b231c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.044+0000 7f01b231c640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:49:18.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.044+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 --> v1:172.21.15.81:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f01ac06b2f0 con 0x7f01ac137940 2024-05-08T15:49:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.045+0000 7f01a8ff9640 1 -- 172.21.15.7:0/1654696348 <== mon.7 v1:172.21.15.81:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 3440275606) 0x7f01ac1cc720 con 0x7f01ac137940 2024-05-08T15:49:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.045+0000 7f01a8ff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:18.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.045+0000 7f01a8ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:18.042 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":20,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:49:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 >> v1:172.21.15.81:6812/847627118 conn(0x7f019803aa30 legacy=0x7f019803cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 10 monclient: shutdown 2024-05-08T15:49:18.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 >> v1:172.21.15.81:6792/0 conn(0x7f01ac137940 legacy=0x7f01ac1d4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 shutdown_connections 2024-05-08T15:49:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 >> 172.21.15.7:0/1654696348 conn(0x7f01ac0a9fa0 msgr2=0x7f01ac1307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 shutdown_connections 2024-05-08T15:49:18.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.047+0000 7f01b231c640 1 -- 172.21.15.7:0/1654696348 wait complete. 2024-05-08T15:49:18.053 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:49:18.054 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T15:49:18.054 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:49:18.054 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:49:18.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.161+0000 7f8817eac640 10 monclient: get_monmap_and_config 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.161+0000 7f8817eac640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.161+0000 7f8817eac640 10 monclient: monmap: 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:18.162588+0000 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:18.162588+0000 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.158 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.159 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.159 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.162+0000 7f8817eac640 1 Processor -- start 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.162+0000 7f8817eac640 1 -- start start 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.162+0000 7f8817eac640 10 monclient: init 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.162+0000 7f8817eac640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.162+0000 7f8817eac640 10 monclient: _add_conns ranks=[2,1,7,5,6,4,0,8,3] 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.162+0000 7f8817eac640 10 monclient(hunting): picked mon.noname-b con 0x7f8810137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 10 monclient(hunting): picked mon.noname-f con 0x7f88100a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 10 monclient(hunting): picked mon.noname-i con 0x7f88100a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88100a2c70 con 0x7f88100a49a0 2024-05-08T15:49:18.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88100a2f50 con 0x7f8810137940 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88100a1670 con 0x7f88100a41b0 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 10 monclient(hunting): authenticate will time out at 966.277704s 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8815c48640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f8810137940 0x7f8810137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43606/0 (socket says 172.21.15.7:43606) 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8815c48640 1 -- 172.21.15.7:0/2754735678 learned_addr learned my addr 172.21.15.7:0/2754735678 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907698060 0 0) 0x7f88100a2f50 con 0x7f8810137940 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): my global_id is 94196 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87fc003520 con 0x7f8810137940 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017949283 0 0) 0x7f88100a1670 con 0x7f88100a41b0 2024-05-08T15:49:18.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): my global_id is 94219 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88100a2f50 con 0x7f88100a41b0 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2781283429 0 0) 0x7f88100a2c70 con 0x7f88100a49a0 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient(hunting): my global_id is 4987 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88100a1670 con 0x7f88100a49a0 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3155179995 0 0) 0x7f87fc003520 con 0x7f8810137940 2024-05-08T15:49:18.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 >> v1:172.21.15.81:6792/0 conn(0x7f88100a41b0 legacy=0x7f8810140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 >> v1:172.21.15.81:6789/0 conn(0x7f88100a49a0 legacy=0x7f881013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 monclient: found mon.noname-b 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88100a39f0 con 0x7f8810137940 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 5 monclient: authenticate success, global_id 94196 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f880c0017b0 con 0x7f8810137940 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: dump: 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.172 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f880c001ad0 con 0x7f8810137940 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 1 -- 172.21.15.7:0/2754735678 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f880c004be0 con 0x7f8810137940 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8817eac640 4 set_mon_vals no callback set 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.163+0000 7f8814c46640 10 monclient: dump: 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.173 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 10 monclient: shutdown 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 1 -- 172.21.15.7:0/2754735678 >> v1:172.21.15.7:6790/0 conn(0x7f8810137940 legacy=0x7f8810137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 1 -- 172.21.15.7:0/2754735678 shutdown_connections 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 1 -- 172.21.15.7:0/2754735678 >> 172.21.15.7:0/2754735678 conn(0x7f88100a9fa0 msgr2=0x7f88100aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 1 -- 172.21.15.7:0/2754735678 shutdown_connections 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 1 -- 172.21.15.7:0/2754735678 wait complete. 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.164+0000 7f8817eac640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient: monmap: 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.175 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 1 Processor -- start 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 1 -- start start 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient: init 2024-05-08T15:49:18.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient: _add_conns ranks=[7,0,2,6,8,5,1,4,3] 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient(hunting): picked mon.noname-i con 0x7f88100a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient(hunting): picked mon.noname-a con 0x7f88100a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient(hunting): picked mon.noname-b con 0x7f8810137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8810133f10 con 0x7f88100a49a0 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88100a0f60 con 0x7f8810137940 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88100a1f30 con 0x7f88100a41b0 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8817eac640 10 monclient(hunting): authenticate will time out at 966.280434s 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8815447640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f88100a49a0 0x7f8810132710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48850/0 (socket says 172.21.15.7:48850) 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.165+0000 7f8815447640 1 -- 172.21.15.7:0/2661651980 learned_addr learned my addr 172.21.15.7:0/2661651980 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3870326172 0 0) 0x7f88100a0f60 con 0x7f8810137940 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): my global_id is 94205 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87f4003520 con 0x7f8810137940 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2828121762 0 0) 0x7f8810133f10 con 0x7f88100a49a0 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): my global_id is 94185 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88100a0f60 con 0x7f88100a49a0 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706783641 0 0) 0x7f88100a1f30 con 0x7f88100a41b0 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): my global_id is 94228 2024-05-08T15:49:18.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8810133f10 con 0x7f88100a41b0 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 236591275 0 0) 0x7f88100a0f60 con 0x7f88100a49a0 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88100a1f30 con 0x7f88100a49a0 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1016143057 0 0) 0x7f87f4003520 con 0x7f8810137940 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f88100a0f60 con 0x7f8810137940 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f88000019d0 con 0x7f88100a49a0 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): dump: 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.179 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8808001dd0 con 0x7f8810137940 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): dump: 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.181 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2155918975 0 0) 0x7f8810133f10 con 0x7f88100a41b0 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87f4003520 con 0x7f88100a41b0 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f880c001c80 con 0x7f88100a41b0 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient(hunting): dump: 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.182 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3493574022 0 0) 0x7f88100a1f30 con 0x7f88100a49a0 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.81:6792/0 conn(0x7f88100a41b0 legacy=0x7f8810132000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.7:6790/0 conn(0x7f8810137940 legacy=0x7f88101d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 monclient: found mon.a 2024-05-08T15:49:18.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8810137550 con 0x7f88100a49a0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8817eac640 5 monclient: authenticate success, global_id 94185 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8817eac640 10 monclient: _renew_subs 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8817eac640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8810132e20 con 0x7f88100a49a0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8817eac640 10 monclient: _renew_subs 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8817eac640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f88101366d0 con 0x7f88100a49a0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f88000038c0 con 0x7f88100a49a0 2024-05-08T15:49:18.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8800003e30 con 0x7f88100a49a0 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f8807fff640 4 set_mon_vals no callback set 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.166+0000 7f88067fc640 10 monclient: dump: 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.185 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.167+0000 7f8817eac640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.167+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f87f80065c0 con 0x7f87f8002860 2024-05-08T15:49:18.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.167+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f8800003fd0 con 0x7f88100a49a0 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.167+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(99..99 src has 1..99) ==== 3506+0+0 (unknown 992023057 0 0) 0x7f8800046b80 con 0x7f88100a49a0 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.167+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1341994591 0 0) 0x7f87f80065c0 con 0x7f87f8002860 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient(hunting): my global_id is 94194 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88100a1f30 con 0x7f87f8002860 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f8816449640 10 monclient: get_auth_request con 0x7f87f403a2b0 auth_method 0 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 677867302 0 0) 0x7f88100a1f30 con 0x7f87f8002860 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87f80065c0 con 0x7f87f8002860 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f880c001ad0 con 0x7f87f8002860 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:18.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient: dump: 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.188 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 377658279 0 0) 0x7f87f80065c0 con 0x7f87f8002860 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f87f800a150 con 0x7f87f8002860 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f880c005180 con 0x7f87f8002860 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.7:6789/0 conn(0x7f87f8002860 legacy=0x7f87f8004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.168+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.7:6789/0 conn(0x7f87f8002860 legacy=0x7f87f8004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f8817eac640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f87f8009e40 con 0x7f87f8009100 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194120003 0 0) 0x7f87f8009e40 con 0x7f87f8009100 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 10 monclient(hunting): my global_id is 94203 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87f80065c0 con 0x7f87f8009100 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 76899508 0 0) 0x7f87f80065c0 con 0x7f87f8009100 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f87f8009e40 con 0x7f87f8009100 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f880c005980 con 0x7f87f8009100 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 10 monclient: dump: 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 21722675 0 0) 0x7f87f8009e40 con 0x7f87f8009100 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.175+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f87f800c3d0 con 0x7f87f8009100 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 446103477) 0x7f880c006d10 con 0x7f87f8009100 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f88067fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f88067fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.7:6789/0 conn(0x7f87f8009100 legacy=0x7f87f8009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f88067fc640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.7:6789/0 conn(0x7f87f8009100 legacy=0x7f87f8009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:18.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 8726, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:18.194 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 3, 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.195 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.196 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.197 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.198 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:18.199 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.201 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.203 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:18.204 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.205 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.206 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.207 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.208 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:18.209 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.210 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.211 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:49:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.212 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:18.212 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.81:6812/847627118 conn(0x7f87f403a2b0 legacy=0x7f87f403c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 10 monclient: shutdown 2024-05-08T15:49:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 >> v1:172.21.15.7:6789/0 conn(0x7f88100a49a0 legacy=0x7f8810132710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8816449640 1 -- 172.21.15.7:0/2661651980 reap_dead start 2024-05-08T15:49:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8816449640 1 -- 172.21.15.7:0/2661651980 reap_dead start 2024-05-08T15:49:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 shutdown_connections 2024-05-08T15:49:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 >> 172.21.15.7:0/2661651980 conn(0x7f88100a9fa0 msgr2=0x7f881009aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 shutdown_connections 2024-05-08T15:49:18.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.176+0000 7f8817eac640 1 -- 172.21.15.7:0/2661651980 wait complete. 2024-05-08T15:49:18.214 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:18.214 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:49:18.215 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:49:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.322+0000 7f9155055640 10 monclient: get_monmap_and_config 2024-05-08T15:49:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.322+0000 7f9155055640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.322+0000 7f9155055640 10 monclient: monmap: 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:18.323320+0000 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:18.323320+0000 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.319 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.320 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.320 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.322+0000 7f9155055640 1 Processor -- start 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.322+0000 7f9155055640 1 -- start start 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.322+0000 7f9155055640 10 monclient: init 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 10 monclient: _add_conns ranks=[4,1,6,7,5,0,2,8,3] 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 10 monclient(hunting): picked mon.noname-c con 0x7f9150137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 10 monclient(hunting): picked mon.noname-f con 0x7f91500a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 10 monclient(hunting): picked mon.noname-d con 0x7f91500a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91500a2c70 con 0x7f91500a49a0 2024-05-08T15:49:18.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91500a2f50 con 0x7f9150137940 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91500a1670 con 0x7f91500a41b0 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f9155055640 10 monclient(hunting): authenticate will time out at 966.438128s 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914ed9d640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f9150137940 0x7f9150137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57950/0 (socket says 172.21.15.7:57950) 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914ed9d640 1 -- 172.21.15.7:0/382168018 learned_addr learned my addr 172.21.15.7:0/382168018 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 357346086 0 0) 0x7f91500a2f50 con 0x7f9150137940 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): my global_id is 5071 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f913c003520 con 0x7f9150137940 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3389100461 0 0) 0x7f91500a1670 con 0x7f91500a41b0 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 10 monclient(hunting): my global_id is 75075 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.323+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91500a2f50 con 0x7f91500a41b0 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787223474 0 0) 0x7f91500a2c70 con 0x7f91500a49a0 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient(hunting): my global_id is 4996 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91500a1670 con 0x7f91500a49a0 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1654153944 0 0) 0x7f913c003520 con 0x7f9150137940 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 >> v1:172.21.15.7:6792/0 conn(0x7f91500a41b0 legacy=0x7f9150140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 >> v1:172.21.15.81:6789/0 conn(0x7f91500a49a0 legacy=0x7f915013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 monclient: found mon.noname-c 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91500a39f0 con 0x7f9150137940 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f91380017b0 con 0x7f9150137940 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 5 monclient: authenticate success, global_id 5071 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: dump: 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.335 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.336 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9138001ad0 con 0x7f9150137940 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 1 -- 172.21.15.7:0/382168018 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9138001d40 con 0x7f9150137940 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f914dd9b640 10 monclient: dump: 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.337 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.338 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 4 set_mon_vals no callback set 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 10 monclient: shutdown 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 1 -- 172.21.15.7:0/382168018 >> v1:172.21.15.7:6791/0 conn(0x7f9150137940 legacy=0x7f9150137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 1 -- 172.21.15.7:0/382168018 shutdown_connections 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 1 -- 172.21.15.7:0/382168018 >> 172.21.15.7:0/382168018 conn(0x7f91500a9fa0 msgr2=0x7f91500aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 1 -- 172.21.15.7:0/382168018 shutdown_connections 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.324+0000 7f9155055640 1 -- 172.21.15.7:0/382168018 wait complete. 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.325+0000 7f9155055640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.325+0000 7f9155055640 10 monclient: monmap: 2024-05-08T15:49:18.339 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.340 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 1 Processor -- start 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 1 -- start start 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient: init 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient: _add_conns ranks=[0,4,2,7,1,3,8,5,6] 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient(hunting): picked mon.noname-a con 0x7f91500a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient(hunting): picked mon.noname-c con 0x7f91500a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient(hunting): picked mon.noname-b con 0x7f9150137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9150132b80 con 0x7f91500a41b0 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9150134de0 con 0x7f9150137940 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91500a1f30 con 0x7f91500a49a0 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f9155055640 10 monclient(hunting): authenticate will time out at 966.441179s 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f914e59c640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f91500a49a0 0x7f9150132470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57958/0 (socket says 172.21.15.7:57958) 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f914e59c640 1 -- 172.21.15.7:0/3474460910 learned_addr learned my addr 172.21.15.7:0/3474460910 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3409016855 0 0) 0x7f91500a1f30 con 0x7f91500a49a0 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): my global_id is 5080 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9130003520 con 0x7f91500a49a0 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4238358683 0 0) 0x7f9150132b80 con 0x7f91500a41b0 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): my global_id is 94212 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91500a1f30 con 0x7f91500a41b0 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558476290 0 0) 0x7f9150134de0 con 0x7f9150137940 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 10 monclient(hunting): my global_id is 94214 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.326+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9150132b80 con 0x7f9150137940 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2073182954 0 0) 0x7f9150132b80 con 0x7f9150137940 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9150134de0 con 0x7f9150137940 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f91480020c0 con 0x7f9150137940 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): dump: 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.344 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2114905201 0 0) 0x7f91500a1f30 con 0x7f91500a41b0 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9150132b80 con 0x7f91500a41b0 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4217863316 0 0) 0x7f9130003520 con 0x7f91500a49a0 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91500a1f30 con 0x7f91500a49a0 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9138001770 con 0x7f91500a41b0 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): dump: 2024-05-08T15:49:18.345 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.346 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f91400019d0 con 0x7f91500a49a0 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient(hunting): dump: 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.347 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 830068180 0 0) 0x7f9150134de0 con 0x7f9150137940 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.7:6791/0 conn(0x7f91500a49a0 legacy=0x7f9150132470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.7:6789/0 conn(0x7f91500a41b0 legacy=0x7f9150131d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 monclient: found mon.b 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91500a0f60 con 0x7f9150137940 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f9155055640 5 monclient: authenticate success, global_id 94214 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f9155055640 10 monclient: _renew_subs 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f9155055640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f91501358d0 con 0x7f9150137940 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f9155055640 10 monclient: _renew_subs 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f9155055640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f91501374f0 con 0x7f9150137940 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9148001950 con 0x7f9150137940 2024-05-08T15:49:18.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9148003f60 con 0x7f9150137940 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f91477fe640 10 monclient: dump: 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.350 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.327+0000 7f914d59a640 4 set_mon_vals no callback set 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f914800f2e0 con 0x7f9150137940 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f9155055640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9150133870 con 0x7f91501343b0 2024-05-08T15:49:18.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(99..99 src has 1..99) ==== 3506+0+0 (unknown 992023057 0 0) 0x7f9148045db0 con 0x7f9150137940 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682941855 0 0) 0x7f91500a39f0 con 0x7f91501343b0 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 10 monclient(hunting): my global_id is 94223 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9150134de0 con 0x7f91501343b0 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4100575190 0 0) 0x7f9150134de0 con 0x7f91501343b0 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91500a39f0 con 0x7f91501343b0 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9138004f60 con 0x7f91501343b0 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f91477fe640 10 monclient: dump: 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.352 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.353 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.328+0000 7f914e59c640 10 monclient: get_auth_request con 0x7f913003a450 auth_method 0 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.329+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3788079396 0 0) 0x7f91500a39f0 con 0x7f91501343b0 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.329+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9150134790 con 0x7f91501343b0 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.329+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9138006460 con 0x7f91501343b0 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.329+0000 7f91477fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.329+0000 7f91477fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.329+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.7:6790/0 conn(0x7f91501343b0 legacy=0x7f915009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.329+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.7:6790/0 conn(0x7f91501343b0 legacy=0x7f915009ba50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.335+0000 7f9155055640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.335+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f91500753d0 con 0x7f915002d050 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320141275 0 0) 0x7f91500753d0 con 0x7f915002d050 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient(hunting): my global_id is 94232 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91500a39f0 con 0x7f915002d050 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1283353133 0 0) 0x7f91500a39f0 con 0x7f915002d050 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91500753d0 con 0x7f915002d050 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9138005590 con 0x7f915002d050 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient: dump: 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.357 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3899707409 0 0) 0x7f91500753d0 con 0x7f915002d050 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f915006b2f0 con 0x7f915002d050 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 1837237706) 0x7f9138005900 con 0x7f915002d050 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.7:6790/0 conn(0x7f915002d050 legacy=0x7f915009c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.336+0000 7f91477fe640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.7:6790/0 conn(0x7f915002d050 legacy=0x7f915009c160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 8887, 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:18.359 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 3, 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.360 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.361 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:18.362 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.363 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.365 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:18.366 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.367 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.368 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:18.369 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.370 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.371 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:18.372 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.373 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.374 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.375 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.376 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.81:6812/847627118 conn(0x7f913003a450 legacy=0x7f913003c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 10 monclient: shutdown 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 >> v1:172.21.15.7:6790/0 conn(0x7f9150137940 legacy=0x7f91501d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f914f59e640 1 -- 172.21.15.7:0/3474460910 reap_dead start 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f914f59e640 1 -- 172.21.15.7:0/3474460910 reap_dead start 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 shutdown_connections 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 >> 172.21.15.7:0/3474460910 conn(0x7f91500a9fa0 msgr2=0x7f9150130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 shutdown_connections 2024-05-08T15:49:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.337+0000 7f9155055640 1 -- 172.21.15.7:0/3474460910 wait complete. 2024-05-08T15:49:18.380 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:18.380 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:49:18.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.488+0000 7f4df4352640 10 monclient: get_monmap_and_config 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.488+0000 7f4df4352640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.488+0000 7f4df4352640 10 monclient: monmap: 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:18.489826+0000 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:18.489826+0000 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.486 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.489+0000 7f4df4352640 1 Processor -- start 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.489+0000 7f4df4352640 1 -- start start 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.489+0000 7f4df4352640 10 monclient: init 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 10 monclient: _add_conns ranks=[6,1,2,3,7,4,8,5,0] 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 10 monclient(hunting): picked mon.noname-d con 0x7f4dec137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 10 monclient(hunting): picked mon.noname-f con 0x7f4dec0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 10 monclient(hunting): picked mon.noname-b con 0x7f4dec0a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec0a2c70 con 0x7f4dec0a49a0 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec0a2f50 con 0x7f4dec0a41b0 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec0a1670 con 0x7f4dec137940 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 10 monclient(hunting): authenticate will time out at 966.604957s 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df20ee640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f4dec137940 0x7f4dec137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35554/0 (socket says 172.21.15.7:35554) 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df20ee640 1 -- 172.21.15.7:0/2408442996 learned_addr learned my addr 172.21.15.7:0/2408442996 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258085387 0 0) 0x7f4dec0a2f50 con 0x7f4dec0a41b0 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): my global_id is 94241 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dd8003520 con 0x7f4dec0a41b0 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111392792 0 0) 0x7f4dec0a1670 con 0x7f4dec137940 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient(hunting): my global_id is 75084 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dec0a2f50 con 0x7f4dec137940 2024-05-08T15:49:18.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1293138227 0 0) 0x7f4dd8003520 con 0x7f4dec0a41b0 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 >> v1:172.21.15.7:6792/0 conn(0x7f4dec137940 legacy=0x7f4dec137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 >> v1:172.21.15.81:6789/0 conn(0x7f4dec0a49a0 legacy=0x7f4dec13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 monclient: found mon.noname-b 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec0a39f0 con 0x7f4dec0a41b0 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4de4001b30 con 0x7f4dec0a41b0 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 5 monclient: authenticate success, global_id 94241 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df10ec640 10 monclient: dump: 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.498 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.490+0000 7f4df4352640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4de4001e10 con 0x7f4dec0a41b0 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df10ec640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df10ec640 1 -- 172.21.15.7:0/2408442996 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4de4003cd0 con 0x7f4dec0a41b0 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 4 set_mon_vals no callback set 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df10ec640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df10ec640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df10ec640 10 monclient: dump: 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.499 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 10 monclient: shutdown 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 1 -- 172.21.15.7:0/2408442996 >> v1:172.21.15.7:6790/0 conn(0x7f4dec0a41b0 legacy=0x7f4dec140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 1 -- 172.21.15.7:0/2408442996 shutdown_connections 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 1 -- 172.21.15.7:0/2408442996 >> 172.21.15.7:0/2408442996 conn(0x7f4dec0a9fa0 msgr2=0x7f4dec0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 1 -- 172.21.15.7:0/2408442996 shutdown_connections 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.491+0000 7f4df4352640 1 -- 172.21.15.7:0/2408442996 wait complete. 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient: monmap: 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.501 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 1 Processor -- start 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 1 -- start start 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient: init 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient: _add_conns ranks=[4,2,1,6,3,0,5,8,7] 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient(hunting): picked mon.noname-c con 0x7f4dec0a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient(hunting): picked mon.noname-b con 0x7f4dec0a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:18.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 10 monclient(hunting): picked mon.noname-f con 0x7f4dec137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.492+0000 7f4df4352640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec137690 con 0x7f4dec137940 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec1358e0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4dec1c9740 con 0x7f4dec0a41b0 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 10 monclient(hunting): authenticate will time out at 966.607658s 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df20ee640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f4dec0a41b0 0x7f4dec140fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57972/0 (socket says 172.21.15.7:57972) 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df20ee640 1 -- 172.21.15.7:0/1389145528 learned_addr learned my addr 172.21.15.7:0/1389145528 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2808428897 0 0) 0x7f4dec1358e0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): my global_id is 94250 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dd0003550 con 0x7f4dec0a49a0 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351053763 0 0) 0x7f4dec1c9740 con 0x7f4dec0a41b0 2024-05-08T15:49:18.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): my global_id is 5089 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dec1358e0 con 0x7f4dec0a41b0 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432144085 0 0) 0x7f4dec137690 con 0x7f4dec137940 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): my global_id is 5014 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dec1c9740 con 0x7f4dec137940 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2943882092 0 0) 0x7f4dec1358e0 con 0x7f4dec0a41b0 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4dec137690 con 0x7f4dec0a41b0 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 555757215 0 0) 0x7f4dd0003550 con 0x7f4dec0a49a0 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4dec1358e0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4de8001970 con 0x7f4dec0a41b0 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.505 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4ddc001a10 con 0x7f4dec0a49a0 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.506 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 69896298 0 0) 0x7f4dec1c9740 con 0x7f4dec137940 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4dd0003550 con 0x7f4dec137940 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2187200795 0 0) 0x7f4dec1358e0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.7:6791/0 conn(0x7f4dec0a41b0 legacy=0x7f4dec140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.81:6789/0 conn(0x7f4dec137940 legacy=0x7f4dec1d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 monclient: found mon.b 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dec0a1f30 con 0x7f4dec0a49a0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 5 monclient: authenticate success, global_id 94250 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 10 monclient: _renew_subs 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4dec0a0f60 con 0x7f4dec0a49a0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 10 monclient: _renew_subs 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f4dec1416e0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4ddc0038c0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4ddc003db0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.493+0000 7f4de2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4df08eb640 4 set_mon_vals no callback set 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 10 monclient: dump: 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.509 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4df4352640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f4ddc00f130 con 0x7f4dec0a49a0 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4dd40065c0 con 0x7f4dd4002860 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(99..99 src has 1..99) ==== 3506+0+0 (unknown 992023057 0 0) 0x7f4ddc0459e0 con 0x7f4dec0a49a0 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150693599 0 0) 0x7f4dd40065c0 con 0x7f4dd4002860 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.494+0000 7f4de2ffd640 10 monclient(hunting): my global_id is 5098 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dec1358e0 con 0x7f4dd4002860 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4df28ef640 10 monclient: get_auth_request con 0x7f4dd003a2c0 auth_method 0 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2414602285 0 0) 0x7f4dec1358e0 con 0x7f4dd4002860 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4dd40065c0 con 0x7f4dd4002860 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4de8004e50 con 0x7f4dd4002860 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 10 monclient: dump: 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.511 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 57483453 0 0) 0x7f4dd40065c0 con 0x7f4dd4002860 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4dd400a150 con 0x7f4dd4002860 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4de8007130 con 0x7f4dd4002860 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:18.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.7:6791/0 conn(0x7f4dd4002860 legacy=0x7f4dd4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.495+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.7:6791/0 conn(0x7f4dd4002860 legacy=0x7f4dd4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4df4352640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4dd4009e20 con 0x7f4dd40090e0 2024-05-08T15:49:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3531740825 0 0) 0x7f4dd4009e20 con 0x7f4dd40090e0 2024-05-08T15:49:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient(hunting): my global_id is 5107 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4dd40065c0 con 0x7f4dd40090e0 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2283160324 0 0) 0x7f4dd40065c0 con 0x7f4dd40090e0 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4dd4009e20 con 0x7f4dd40090e0 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4de80057d0 con 0x7f4dd40090e0 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient: dump: 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.515 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 934113872 0 0) 0x7f4dd4009e20 con 0x7f4dd40090e0 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4dd400c390 con 0x7f4dd40090e0 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 3537342654) 0x7f4de800a130 con 0x7f4dd40090e0 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:18.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.7:6791/0 conn(0x7f4dd40090e0 legacy=0x7f4dd4009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.502+0000 7f4de2ffd640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.7:6791/0 conn(0x7f4dd40090e0 legacy=0x7f4dd4009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 150810, 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:18.517 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.518 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.519 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:18.520 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.523 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.525 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.526 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.528 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.530 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.531 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.532 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:18.533 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:18.534 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.81:6812/847627118 conn(0x7f4dd003a2c0 legacy=0x7f4dd003c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 10 monclient: shutdown 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 >> v1:172.21.15.7:6790/0 conn(0x7f4dec0a49a0 legacy=0x7f4dec1d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df28ef640 1 -- 172.21.15.7:0/1389145528 reap_dead start 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df28ef640 1 -- 172.21.15.7:0/1389145528 reap_dead start 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 shutdown_connections 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 >> 172.21.15.7:0/1389145528 conn(0x7f4dec0a9fa0 msgr2=0x7f4dec130760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 shutdown_connections 2024-05-08T15:49:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.503+0000 7f4df4352640 1 -- 172.21.15.7:0/1389145528 wait complete. 2024-05-08T15:49:18.537 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:18.537 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:49:18.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:49:18.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.652+0000 7fb4c2638640 10 monclient: get_monmap_and_config 2024-05-08T15:49:18.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.652+0000 7fb4c2638640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.652+0000 7fb4c2638640 10 monclient: monmap: 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:18.653633+0000 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:18.653633+0000 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.658 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 1 Processor -- start 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 1 -- start start 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient: init 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient: _add_conns ranks=[6,1,8,3,4,0,7,5,2] 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient(hunting): picked mon.noname-d con 0x7fb4bc137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient(hunting): picked mon.noname-f con 0x7fb4bc0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient(hunting): picked mon.noname-e con 0x7fb4bc0a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc0a2c70 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc0a2f50 con 0x7fb4bc137940 2024-05-08T15:49:18.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc0a1670 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.653+0000 7fb4c2638640 10 monclient(hunting): authenticate will time out at 966.768586s 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4bbfff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fb4bc137940 0x7fb4bc137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35562/0 (socket says 172.21.15.7:35562) 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4bbfff640 1 -- 172.21.15.7:0/2855467914 learned_addr learned my addr 172.21.15.7:0/2855467914 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4c0bd5640 1 --1- 172.21.15.7:0/2855467914 >> v1:172.21.15.7:6793/0 conn(0x7fb4bc0a41b0 0x7fb4bc140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:57830/0 (socket says 172.21.15.7:57830) 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1784744994 0 0) 0x7fb4bc0a1670 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): my global_id is 94202 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a4003520 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2477050903 0 0) 0x7fb4bc0a2f50 con 0x7fb4bc137940 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): my global_id is 75093 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4bc0a1670 con 0x7fb4bc137940 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 748642806 0 0) 0x7fb4bc0a2c70 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient(hunting): my global_id is 5023 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4bc0a2f50 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1005319050 0 0) 0x7fb4bc0a1670 con 0x7fb4bc137940 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 >> v1:172.21.15.7:6793/0 conn(0x7fb4bc0a41b0 legacy=0x7fb4bc140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 >> v1:172.21.15.81:6789/0 conn(0x7fb4bc0a49a0 legacy=0x7fb4bc13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 monclient: found mon.noname-d 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc0a39f0 con 0x7fb4bc137940 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4c2638640 5 monclient: authenticate success, global_id 75093 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4ac001710 con 0x7fb4bc137940 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: dump: 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.662 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4c2638640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb4ac0019f0 con 0x7fb4bc137940 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4c2638640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 1 -- 172.21.15.7:0/2855467914 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4ac004be0 con 0x7fb4bc137940 2024-05-08T15:49:18.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4c2638640 4 set_mon_vals no callback set 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.654+0000 7fb4baffd640 10 monclient: dump: 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.664 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 10 monclient: shutdown 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 1 -- 172.21.15.7:0/2855467914 >> v1:172.21.15.7:6792/0 conn(0x7fb4bc137940 legacy=0x7fb4bc137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 1 -- 172.21.15.7:0/2855467914 shutdown_connections 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 1 -- 172.21.15.7:0/2855467914 >> 172.21.15.7:0/2855467914 conn(0x7fb4bc0a9fa0 msgr2=0x7fb4bc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 1 -- 172.21.15.7:0/2855467914 shutdown_connections 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 1 -- 172.21.15.7:0/2855467914 wait complete. 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.655+0000 7fb4c2638640 10 monclient: monmap: 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.665 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.666 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 1 Processor -- start 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 1 -- start start 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 10 monclient: init 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 10 monclient: _add_conns ranks=[2,0,5,1,4,8,3,7,6] 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 10 monclient(hunting): picked mon.noname-b con 0x7fb4bc0a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 10 monclient(hunting): picked mon.noname-a con 0x7fb4bc0a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 10 monclient(hunting): picked mon.noname-h con 0x7fb4bc137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc137730 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc0a1f30 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb4bc0a0f60 con 0x7fb4bc137940 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.656+0000 7fb4c2638640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 10 monclient(hunting): authenticate will time out at 966.771614s 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4bbfff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fb4bc0a41b0 0x7fb4bc137020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43666/0 (socket says 172.21.15.7:43666) 2024-05-08T15:49:18.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4bbfff640 1 -- 172.21.15.7:0/3860310084 learned_addr learned my addr 172.21.15.7:0/3860310084 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4bb7fe640 1 --1- 172.21.15.7:0/3860310084 >> v1:172.21.15.7:6789/0 conn(0x7fb4bc0a49a0 0x7fb4bc1d16f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48876/0 (socket says 172.21.15.7:48876) 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328003712 0 0) 0x7fb4bc0a1f30 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): my global_id is 94259 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4a8003520 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 976302094 0 0) 0x7fb4bc137730 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): my global_id is 94221 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4bc0a1f30 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1827173197 0 0) 0x7fb4bc0a0f60 con 0x7fb4bc137940 2024-05-08T15:49:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): my global_id is 5063 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4bc137730 con 0x7fb4bc137940 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2580641518 0 0) 0x7fb4a8003520 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4bc0a0f60 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4ac004b20 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4283684855 0 0) 0x7fb4bc0a1f30 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4a8003520 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4b0001a50 con 0x7fb4bc0a49a0 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 168972066 0 0) 0x7fb4bc137730 con 0x7fb4bc137940 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb4bc0a1f30 con 0x7fb4bc137940 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4b4001750 con 0x7fb4bc137940 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient(hunting): dump: 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1838198112 0 0) 0x7fb4bc0a0f60 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.81:6791/0 conn(0x7fb4bc137940 legacy=0x7fb4bc1d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.7:6789/0 conn(0x7fb4bc0a49a0 legacy=0x7fb4bc1d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 monclient: found mon.b 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc134d90 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 5 monclient: authenticate success, global_id 94259 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 10 monclient: _renew_subs 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc135af0 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 10 monclient: _renew_subs 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:18.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fb4bc134760 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb4ac0019f0 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4ac0016f0 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4ba7fc640 4 set_mon_vals no callback set 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.657+0000 7fb4b8ff9640 10 monclient: dump: 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.675 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.658+0000 7fb4c2638640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.658+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fb4ac010370 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.658+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb49c0065c0 con 0x7fb49c002860 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(100..100 src has 1..100) ==== 3898+0+0 (unknown 251748867 0 0) 0x7fb4ac046e60 con 0x7fb4bc0a41b0 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1541888856 0 0) 0x7fb49c0065c0 con 0x7fb49c002860 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient(hunting): my global_id is 75102 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4bc0a0f60 con 0x7fb49c002860 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4c0bd5640 10 monclient: get_auth_request con 0x7fb4a803a4a0 auth_method 0 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 863725189 0 0) 0x7fb4bc0a0f60 con 0x7fb49c002860 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb49c0065c0 con 0x7fb49c002860 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4b0003b40 con 0x7fb49c002860 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient: dump: 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.677 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3121096824 0 0) 0x7fb49c0065c0 con 0x7fb49c002860 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb49c00a150 con 0x7fb49c002860 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb4b00057c0 con 0x7fb49c002860 2024-05-08T15:49:18.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:18.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.7:6792/0 conn(0x7fb49c002860 legacy=0x7fb49c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.659+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.7:6792/0 conn(0x7fb49c002860 legacy=0x7fb49c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4c2638640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:18.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb49c009e20 con 0x7fb49c0090e0 2024-05-08T15:49:18.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2512168727 0 0) 0x7fb49c009e20 con 0x7fb49c0090e0 2024-05-08T15:49:18.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 10 monclient(hunting): my global_id is 75111 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb49c0065c0 con 0x7fb49c0090e0 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1452732676 0 0) 0x7fb49c0065c0 con 0x7fb49c0090e0 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb49c009e20 con 0x7fb49c0090e0 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb4b0004330 con 0x7fb49c0090e0 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.666+0000 7fb4b8ff9640 10 monclient: dump: 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.681 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3369390284 0 0) 0x7fb49c009e20 con 0x7fb49c0090e0 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb49c00c390 con 0x7fb49c0090e0 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7863 (unknown 0 0 1765973485) 0x7fb4b00079b0 con 0x7fb49c0090e0 2024-05-08T15:49:18.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4b8ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4b8ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.7:6792/0 conn(0x7fb49c0090e0 legacy=0x7fb49c009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4b8ff9640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.7:6792/0 conn(0x7fb49c0090e0 legacy=0x7fb49c009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 150981, 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:18.683 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.684 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.685 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:18.686 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.687 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:18.688 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:18.689 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.690 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.691 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.692 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:18.693 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.694 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:18.695 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.698 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:18.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.700 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.701 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.701 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.701 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.701 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:18.701 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.81:6812/847627118 conn(0x7fb4a803a4a0 legacy=0x7fb4a803c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 10 monclient: shutdown 2024-05-08T15:49:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 >> v1:172.21.15.7:6790/0 conn(0x7fb4bc0a41b0 legacy=0x7fb4bc137020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c0bd5640 1 -- 172.21.15.7:0/3860310084 reap_dead start 2024-05-08T15:49:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c0bd5640 1 -- 172.21.15.7:0/3860310084 reap_dead start 2024-05-08T15:49:18.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 shutdown_connections 2024-05-08T15:49:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 >> 172.21.15.7:0/3860310084 conn(0x7fb4bc0a9fa0 msgr2=0x7fb4bc130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 shutdown_connections 2024-05-08T15:49:18.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.667+0000 7fb4c2638640 1 -- 172.21.15.7:0/3860310084 wait complete. 2024-05-08T15:49:18.703 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:18.704 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:49:18.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.811+0000 7f52a5627640 10 monclient: get_monmap_and_config 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.811+0000 7f52a5627640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.811+0000 7f52a5627640 10 monclient: monmap: 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:18.812888+0000 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:18.812888+0000 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.808 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.809 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.812+0000 7f52a5627640 1 Processor -- start 2024-05-08T15:49:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.812+0000 7f52a5627640 1 -- start start 2024-05-08T15:49:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.812+0000 7f52a5627640 10 monclient: init 2024-05-08T15:49:18.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient: _add_conns ranks=[4,7,5,8,6,1,0,3,2] 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient(hunting): picked mon.noname-c con 0x7f52a0137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient(hunting): picked mon.noname-i con 0x7f52a00a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient(hunting): picked mon.noname-h con 0x7f52a00a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a00a2c70 con 0x7f52a0137940 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a00a2f50 con 0x7f52a00a41b0 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a00a1670 con 0x7f52a00a49a0 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient(hunting): authenticate will time out at 966.927744s 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529effd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f52a0137940 0x7f52a0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57984/0 (socket says 172.21.15.7:57984) 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529effd640 1 -- 172.21.15.7:0/1314213774 learned_addr learned my addr 172.21.15.7:0/1314213774 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3906085560 0 0) 0x7f52a00a2c70 con 0x7f52a0137940 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient(hunting): my global_id is 5116 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f528c003520 con 0x7f52a0137940 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1432767027 0 0) 0x7f528c003520 con 0x7f52a0137940 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 >> v1:172.21.15.81:6792/0 conn(0x7f52a00a49a0 legacy=0x7f52a013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 >> v1:172.21.15.81:6791/0 conn(0x7f52a00a41b0 legacy=0x7f52a0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 monclient: found mon.noname-c 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a00a39f0 con 0x7f52a0137940 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f52980017b0 con 0x7f52a0137940 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 5 monclient: authenticate success, global_id 5116 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: dump: 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.820 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5298001b10 con 0x7f52a0137940 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 1 -- 172.21.15.7:0/1314213774 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5298004be0 con 0x7f52a0137940 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f52a5627640 4 set_mon_vals no callback set 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.813+0000 7f529dffb640 10 monclient: dump: 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.821 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.822 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.814+0000 7f52a5627640 10 monclient: shutdown 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.814+0000 7f52a5627640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.814+0000 7f52a5627640 1 -- 172.21.15.7:0/1314213774 >> v1:172.21.15.7:6791/0 conn(0x7f52a0137940 legacy=0x7f52a0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.814+0000 7f52a5627640 1 -- 172.21.15.7:0/1314213774 shutdown_connections 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.814+0000 7f52a5627640 1 -- 172.21.15.7:0/1314213774 >> 172.21.15.7:0/1314213774 conn(0x7f52a00a9fa0 msgr2=0x7f52a00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.814+0000 7f52a5627640 1 -- 172.21.15.7:0/1314213774 shutdown_connections 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.814+0000 7f52a5627640 1 -- 172.21.15.7:0/1314213774 wait complete. 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.815+0000 7f52a5627640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.815+0000 7f52a5627640 10 monclient: monmap: 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.823 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.815+0000 7f52a5627640 1 Processor -- start 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.815+0000 7f52a5627640 1 -- start start 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.815+0000 7f52a5627640 10 monclient: init 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.815+0000 7f52a5627640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.815+0000 7f52a5627640 10 monclient: _add_conns ranks=[6,4,7,0,3,8,5,1,2] 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 10 monclient(hunting): picked mon.noname-d con 0x7f52a00a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 10 monclient(hunting): picked mon.noname-c con 0x7f52a00a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 10 monclient(hunting): picked mon.noname-i con 0x7f52a0137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a009d5f0 con 0x7f52a00a49a0 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a009f8e0 con 0x7f52a00a41b0 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a009da90 con 0x7f52a0137940 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 10 monclient(hunting): authenticate will time out at 966.930740s 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f529e7fc640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f52a00a49a0 0x7f52a009c030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:57998/0 (socket says 172.21.15.7:57998) 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f529e7fc640 1 -- 172.21.15.7:0/4042769604 learned_addr learned my addr 172.21.15.7:0/4042769604 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3398620112 0 0) 0x7f52a009d5f0 con 0x7f52a00a49a0 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): my global_id is 5125 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5280003520 con 0x7f52a00a49a0 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501745671 0 0) 0x7f52a009f8e0 con 0x7f52a00a41b0 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): my global_id is 75120 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52a009d5f0 con 0x7f52a00a41b0 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2443402653 0 0) 0x7f52a009da90 con 0x7f52a0137940 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): my global_id is 94246 2024-05-08T15:49:18.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52a009f8e0 con 0x7f52a0137940 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2425135143 0 0) 0x7f5280003520 con 0x7f52a00a49a0 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f52a009da90 con 0x7f52a00a49a0 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3318165156 0 0) 0x7f52a009d5f0 con 0x7f52a00a41b0 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5280003520 con 0x7f52a00a41b0 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5288001b20 con 0x7f52a00a49a0 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): dump: 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.827 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2968432783 0 0) 0x7f52a009f8e0 con 0x7f52a0137940 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f52a009d5f0 con 0x7f52a0137940 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5298001770 con 0x7f52a00a41b0 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): dump: 2024-05-08T15:49:18.828 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.829 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5290001cd0 con 0x7f52a0137940 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient(hunting): dump: 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.830 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2047168768 0 0) 0x7f52a009da90 con 0x7f52a00a49a0 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.81:6792/0 conn(0x7f52a0137940 legacy=0x7f52a01d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.7:6792/0 conn(0x7f52a00a41b0 legacy=0x7f52a009b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 monclient: found mon.c 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a00a1f30 con 0x7f52a00a49a0 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52977fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 5 monclient: authenticate success, global_id 5125 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 10 monclient: _renew_subs 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.816+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f52a00a0f60 con 0x7f52a00a49a0 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52a5627640 10 monclient: _renew_subs 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52a5627640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f52a009f600 con 0x7f52a00a49a0 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f52880038c0 con 0x7f52a00a49a0 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52977fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5288003db0 con 0x7f52a00a49a0 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52977fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52977fe640 10 monclient: dump: 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.832 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f529d7fa640 4 set_mon_vals no callback set 2024-05-08T15:49:18.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52a5627640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f52840065c0 con 0x7f5284002860 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.817+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f5288003fd0 con 0x7f52a00a49a0 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(100..100 src has 1..100) ==== 3898+0+0 (unknown 251748867 0 0) 0x7f5288046fa0 con 0x7f52a00a49a0 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586948109 0 0) 0x7f52840065c0 con 0x7f5284002860 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient(hunting): my global_id is 94211 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52a009da90 con 0x7f5284002860 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f529f7fe640 10 monclient: get_auth_request con 0x7f528003a410 auth_method 0 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2318542066 0 0) 0x7f52a009da90 con 0x7f5284002860 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f52840065c0 con 0x7f5284002860 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5298004b40 con 0x7f5284002860 2024-05-08T15:49:18.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient: dump: 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.835 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 270432949 0 0) 0x7f52840065c0 con 0x7f5284002860 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f528400a150 con 0x7f5284002860 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5298007140 con 0x7f5284002860 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.7:6793/0 conn(0x7f5284002860 legacy=0x7f5284004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.818+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.7:6793/0 conn(0x7f5284002860 legacy=0x7f5284004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52a5627640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5284009e40 con 0x7f5284009100 2024-05-08T15:49:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3125356325 0 0) 0x7f5284009e40 con 0x7f5284009100 2024-05-08T15:49:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 10 monclient(hunting): my global_id is 94220 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52840065c0 con 0x7f5284009100 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2489711747 0 0) 0x7f52840065c0 con 0x7f5284009100 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5284009e40 con 0x7f5284009100 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f52980058a0 con 0x7f5284009100 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.825+0000 7f52977fe640 10 monclient: dump: 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.838 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2023618593 0 0) 0x7f5284009e40 con 0x7f5284009100 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f528400c3d0 con 0x7f5284009100 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 4248955206) 0x7f529800a090 con 0x7f5284009100 2024-05-08T15:49:18.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52977fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52977fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.7:6793/0 conn(0x7f5284009100 legacy=0x7f5284009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52977fe640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.7:6793/0 conn(0x7f5284009100 legacy=0x7f5284009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 9378, 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:18.840 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.841 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.842 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:18.843 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:18.844 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:18.845 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.846 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:18.847 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.848 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:18.849 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.850 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:18.851 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.852 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.853 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.854 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:18.857 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:18.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.81:6812/847627118 conn(0x7f528003a410 legacy=0x7f528003c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 10 monclient: shutdown 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 >> v1:172.21.15.7:6791/0 conn(0x7f52a00a49a0 legacy=0x7f52a009c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f529f7fe640 1 -- 172.21.15.7:0/4042769604 reap_dead start 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f529f7fe640 1 -- 172.21.15.7:0/4042769604 reap_dead start 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 shutdown_connections 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 >> 172.21.15.7:0/4042769604 conn(0x7f52a00a9fa0 msgr2=0x7f52a00a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 shutdown_connections 2024-05-08T15:49:18.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.826+0000 7f52a5627640 1 -- 172.21.15.7:0/4042769604 wait complete. 2024-05-08T15:49:18.859 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:18.860 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:49:18.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:49:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.968+0000 7f282c278640 10 monclient: get_monmap_and_config 2024-05-08T15:49:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.968+0000 7f282c278640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.968+0000 7f282c278640 10 monclient: monmap: 2024-05-08T15:49:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:18.969018+0000 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:18.969018+0000 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.965 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.966 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.968+0000 7f282c278640 1 Processor -- start 2024-05-08T15:49:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.968+0000 7f282c278640 1 -- start start 2024-05-08T15:49:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.968+0000 7f282c278640 10 monclient: init 2024-05-08T15:49:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 10 monclient: _add_conns ranks=[0,1,5,6,3,8,7,4,2] 2024-05-08T15:49:18.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 10 monclient(hunting): picked mon.noname-a con 0x7f28240a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 10 monclient(hunting): picked mon.noname-f con 0x7f28240a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 10 monclient(hunting): picked mon.noname-h con 0x7f282413e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28240a1660 con 0x7f28240a53a0 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28240a28f0 con 0x7f28240a5780 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28240a2bd0 con 0x7f282413e310 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282c278640 10 monclient(hunting): authenticate will time out at 967.083998s 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282a014640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f28240a53a0 0x7f28240a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48888/0 (socket says 172.21.15.7:48888) 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f282a014640 1 -- 172.21.15.7:0/3251543573 learned_addr learned my addr 172.21.15.7:0/3251543573 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688105075 0 0) 0x7f28240a1660 con 0x7f28240a53a0 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): my global_id is 94230 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2810003520 con 0x7f28240a53a0 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1040994652 0 0) 0x7f28240a28f0 con 0x7f28240a5780 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): my global_id is 5032 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28240a1660 con 0x7f28240a5780 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 581383026 0 0) 0x7f28240a2bd0 con 0x7f282413e310 2024-05-08T15:49:18.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.969+0000 7f2829012640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient(hunting): my global_id is 5081 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28240a28f0 con 0x7f282413e310 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 158706365 0 0) 0x7f2810003520 con 0x7f28240a53a0 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 >> v1:172.21.15.81:6791/0 conn(0x7f282413e310 legacy=0x7f2824140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 >> v1:172.21.15.81:6789/0 conn(0x7f28240a5780 legacy=0x7f282413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 monclient: found mon.noname-a 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28240a6730 con 0x7f28240a53a0 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 5 monclient: authenticate success, global_id 94230 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f28200017b0 con 0x7f28240a53a0 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: dump: 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.984 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2820001a10 con 0x7f28240a53a0 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 1 -- 172.21.15.7:0/3251543573 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2820004be0 con 0x7f28240a53a0 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 4 set_mon_vals no callback set 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f2829012640 10 monclient: dump: 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.986 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 10 monclient: shutdown 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 1 -- 172.21.15.7:0/3251543573 >> v1:172.21.15.7:6789/0 conn(0x7f28240a53a0 legacy=0x7f28240a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 1 -- 172.21.15.7:0/3251543573 shutdown_connections 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 1 -- 172.21.15.7:0/3251543573 >> 172.21.15.7:0/3251543573 conn(0x7f28240a9fa0 msgr2=0x7f28240aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 1 -- 172.21.15.7:0/3251543573 shutdown_connections 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.970+0000 7f282c278640 1 -- 172.21.15.7:0/3251543573 wait complete. 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.971+0000 7f282c278640 10 monclient: build_initial_monmap 2024-05-08T15:49:18.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.971+0000 7f282c278640 10 monclient: monmap: 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:18.988 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.971+0000 7f282c278640 1 Processor -- start 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.971+0000 7f282c278640 1 -- start start 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.971+0000 7f282c278640 10 monclient: init 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 10 monclient: _add_conns ranks=[0,8,1,6,3,7,5,4,2] 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 10 monclient(hunting): picked mon.noname-a con 0x7f28240a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 10 monclient(hunting): picked mon.noname-e con 0x7f28240a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:18.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 10 monclient(hunting): picked mon.noname-f con 0x7f282413e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28241369b0 con 0x7f28240a53a0 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2824136c90 con 0x7f282413e310 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28240a43c0 con 0x7f28240a5780 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282c278640 10 monclient(hunting): authenticate will time out at 967.087008s 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282a014640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f28240a53a0 0x7f2824133070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48898/0 (socket says 172.21.15.7:48898) 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f282a014640 1 -- 172.21.15.7:0/3953600238 learned_addr learned my addr 172.21.15.7:0/3953600238 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2169955740 0 0) 0x7f28241369b0 con 0x7f28240a53a0 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): my global_id is 94239 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2808003520 con 0x7f28240a53a0 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016423045 0 0) 0x7f28240a43c0 con 0x7f28240a5780 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): my global_id is 94229 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28241369b0 con 0x7f28240a5780 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2568506207 0 0) 0x7f2824136c90 con 0x7f282413e310 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): my global_id is 5041 2024-05-08T15:49:18.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f28240a43c0 con 0x7f282413e310 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2468421476 0 0) 0x7f2808003520 con 0x7f28240a53a0 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2824136c90 con 0x7f28240a53a0 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2820004b20 con 0x7f28240a53a0 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): dump: 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.992 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.993 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2531839090 0 0) 0x7f28241369b0 con 0x7f28240a5780 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2808003520 con 0x7f28240a5780 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f28140019d0 con 0x7f28240a5780 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 10 monclient(hunting): dump: 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.994 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.972+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3195935011 0 0) 0x7f28240a43c0 con 0x7f282413e310 2024-05-08T15:49:18.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f28241369b0 con 0x7f282413e310 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f281c001d30 con 0x7f282413e310 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient(hunting): dump: 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:18.996 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3406143927 0 0) 0x7f2824136c90 con 0x7f28240a53a0 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.7:6793/0 conn(0x7f28240a5780 legacy=0x7f2824133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.81:6789/0 conn(0x7f282413e310 legacy=0x7f28241d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:18.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 monclient: found mon.a 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2824134c10 con 0x7f28240a53a0 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: _finish_auth 0 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 5 monclient: authenticate success, global_id 94239 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 10 monclient: _renew_subs 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2824133e90 con 0x7f28240a53a0 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 10 monclient: _renew_subs 2024-05-08T15:49:18.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f28241354b0 con 0x7f28240a53a0 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2820001a10 con 0x7f28240a53a0 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2820001110 con 0x7f28240a53a0 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f2828811640 4 set_mon_vals no callback set 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f281affd640 10 monclient: dump: 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:18.999 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.000 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.973+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f280c0065c0 con 0x7f280c002860 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2820010410 con 0x7f28240a53a0 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(100..100 src has 1..100) ==== 3898+0+0 (unknown 251748867 0 0) 0x7f2820046e30 con 0x7f28240a53a0 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 608862591 0 0) 0x7f280c0065c0 con 0x7f280c002860 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 10 monclient(hunting): my global_id is 5050 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2824136c90 con 0x7f280c002860 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f282a815640 10 monclient: get_auth_request con 0x7f280803a3d0 auth_method 0 2024-05-08T15:49:19.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3678756052 0 0) 0x7f2824136c90 con 0x7f280c002860 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f280c0065c0 con 0x7f280c002860 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2814003930 con 0x7f280c002860 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 10 monclient: dump: 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.002 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.003 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2870147849 0 0) 0x7f280c0065c0 con 0x7f280c002860 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.974+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f280c00a150 con 0x7f280c002860 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.975+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2814003e30 con 0x7f280c002860 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.975+0000 7f281affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.975+0000 7f281affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.975+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.81:6789/0 conn(0x7f280c002860 legacy=0x7f280c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.975+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.81:6789/0 conn(0x7f280c002860 legacy=0x7f280c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.981+0000 7f282c278640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.981+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f280c009e20 con 0x7f280c0090e0 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 192002990 0 0) 0x7f280c009e20 con 0x7f280c0090e0 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 10 monclient(hunting): my global_id is 5059 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f280c0065c0 con 0x7f280c0090e0 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1424775731 0 0) 0x7f280c0065c0 con 0x7f280c0090e0 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f280c009e20 con 0x7f280c0090e0 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2814004550 con 0x7f280c0090e0 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 10 monclient: dump: 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1537944262 0 0) 0x7f280c009e20 con 0x7f280c0090e0 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.982+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f280c00c390 con 0x7f280c0090e0 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7695 (unknown 0 0 3665230348) 0x7f2814005910 con 0x7f280c0090e0 2024-05-08T15:49:19.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f281affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f281affd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.81:6789/0 conn(0x7f280c0090e0 legacy=0x7f280c009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f281affd640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.81:6789/0 conn(0x7f280c0090e0 legacy=0x7f280c009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:49:19.008 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 151298, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:19.009 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.011 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:19.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.017 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.018 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.020 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.021 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:19.022 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:19.023 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.024 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.025 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:19.026 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.027 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:19.028 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:19.029 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.030 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:19.031 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.81:6812/847627118 conn(0x7f280803a3d0 legacy=0x7f280803c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 10 monclient: shutdown 2024-05-08T15:49:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 >> v1:172.21.15.7:6789/0 conn(0x7f28240a53a0 legacy=0x7f2824133070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282a815640 1 -- 172.21.15.7:0/3953600238 reap_dead start 2024-05-08T15:49:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282a815640 1 -- 172.21.15.7:0/3953600238 reap_dead start 2024-05-08T15:49:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 shutdown_connections 2024-05-08T15:49:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 >> 172.21.15.7:0/3953600238 conn(0x7f28240a9fa0 msgr2=0x7f28240a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 shutdown_connections 2024-05-08T15:49:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:18.983+0000 7f282c278640 1 -- 172.21.15.7:0/3953600238 wait complete. 2024-05-08T15:49:19.034 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:19.034 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:49:19.034 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:49:19.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.143+0000 7fb9a1006640 10 monclient: get_monmap_and_config 2024-05-08T15:49:19.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.143+0000 7fb9a1006640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.143+0000 7fb9a1006640 10 monclient: monmap: 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:19.144139+0000 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:19.144139+0000 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.140 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.141 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.141 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.143+0000 7fb9a1006640 1 Processor -- start 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.143+0000 7fb9a1006640 1 -- start start 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.143+0000 7fb9a1006640 10 monclient: init 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 10 monclient: _add_conns ranks=[0,1,6,3,4,5,7,2,8] 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 10 monclient(hunting): picked mon.noname-a con 0x7fb99c0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 10 monclient(hunting): picked mon.noname-f con 0x7fb99c0a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 10 monclient(hunting): picked mon.noname-d con 0x7fb99c13e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:19.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c0a1660 con 0x7fb99c0a53a0 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c0a28f0 con 0x7fb99c0a5780 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c0a2bd0 con 0x7fb99c13e310 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb9a1006640 10 monclient(hunting): authenticate will time out at 967.259003s 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb99b59e640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fb99c13e310 0x7fb99c140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35612/0 (socket says 172.21.15.7:35612) 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb99b59e640 1 -- 172.21.15.7:0/2796615927 learned_addr learned my addr 172.21.15.7:0/2796615927 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb99ad9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb99c0a53a0 0x7fb99c0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48914/0 (socket says 172.21.15.7:48914) 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925856544 0 0) 0x7fb99c0a1660 con 0x7fb99c0a53a0 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): my global_id is 94248 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb988003520 con 0x7fb99c0a53a0 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2128137368 0 0) 0x7fb99c0a2bd0 con 0x7fb99c13e310 2024-05-08T15:49:19.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): my global_id is 75129 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb99c0a1660 con 0x7fb99c13e310 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2952124480 0 0) 0x7fb99c0a28f0 con 0x7fb99c0a5780 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 10 monclient(hunting): my global_id is 5068 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb99c0a2bd0 con 0x7fb99c0a5780 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1247491615 0 0) 0x7fb988003520 con 0x7fb99c0a53a0 2024-05-08T15:49:19.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.144+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 >> v1:172.21.15.7:6792/0 conn(0x7fb99c13e310 legacy=0x7fb99c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 >> v1:172.21.15.81:6789/0 conn(0x7fb99c0a5780 legacy=0x7fb99c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: _finish_hunting 0 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 1 monclient: found mon.noname-a 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c0a6730 con 0x7fb99c0a53a0 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: _finish_auth 0 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: _check_auth_tickets 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb994001a60 con 0x7fb99c0a53a0 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 5 monclient: authenticate success, global_id 94248 2024-05-08T15:49:19.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: dump: 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.156 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb994001d80 con 0x7fb99c0a53a0 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 1 -- 172.21.15.7:0/2796615927 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb994004ea0 con 0x7fb99c0a53a0 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 4 set_mon_vals no callback set 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb999d9b640 10 monclient: dump: 2024-05-08T15:49:19.157 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.158 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 10 monclient: shutdown 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 1 -- 172.21.15.7:0/2796615927 >> v1:172.21.15.7:6789/0 conn(0x7fb99c0a53a0 legacy=0x7fb99c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 1 -- 172.21.15.7:0/2796615927 shutdown_connections 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 1 -- 172.21.15.7:0/2796615927 >> 172.21.15.7:0/2796615927 conn(0x7fb99c0a9fa0 msgr2=0x7fb99c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 1 -- 172.21.15.7:0/2796615927 shutdown_connections 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.145+0000 7fb9a1006640 1 -- 172.21.15.7:0/2796615927 wait complete. 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.146+0000 7fb9a1006640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.146+0000 7fb9a1006640 10 monclient: monmap: 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:19.159 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.160 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.146+0000 7fb9a1006640 1 Processor -- start 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 1 -- start start 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient: init 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient: _add_conns ranks=[0,1,8,5,6,7,3,2,4] 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient(hunting): picked mon.noname-a con 0x7fb99c0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient(hunting): picked mon.noname-f con 0x7fb99c0a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient(hunting): picked mon.noname-e con 0x7fb99c13e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c1360e0 con 0x7fb99c0a53a0 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c137600 con 0x7fb99c0a5780 2024-05-08T15:49:19.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb99c0a43c0 con 0x7fb99c13e310 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9a1006640 10 monclient(hunting): authenticate will time out at 967.262078s 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb99ad9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fb99c0a53a0 0x7fb99c131b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48920/0 (socket says 172.21.15.7:48920) 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb99ad9d640 1 -- 172.21.15.7:0/568949988 learned_addr learned my addr 172.21.15.7:0/568949988 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1381953987 0 0) 0x7fb99c1360e0 con 0x7fb99c0a53a0 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): my global_id is 94257 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb97c003520 con 0x7fb99c0a53a0 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 584528333 0 0) 0x7fb99c0a43c0 con 0x7fb99c13e310 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 10 monclient(hunting): my global_id is 94238 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.147+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb99c1360e0 con 0x7fb99c13e310 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1438287887 0 0) 0x7fb99c137600 con 0x7fb99c0a5780 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): my global_id is 5077 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb99c0a43c0 con 0x7fb99c0a5780 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 755487554 0 0) 0x7fb97c003520 con 0x7fb99c0a53a0 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb99c137600 con 0x7fb99c0a53a0 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb994001a20 con 0x7fb99c0a53a0 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): dump: 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.163 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:19.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3447883446 0 0) 0x7fb99c1360e0 con 0x7fb99c13e310 2024-05-08T15:49:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb97c003520 con 0x7fb99c13e310 2024-05-08T15:49:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb98c001890 con 0x7fb99c13e310 2024-05-08T15:49:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): dump: 2024-05-08T15:49:19.165 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.166 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1414678086 0 0) 0x7fb99c0a43c0 con 0x7fb99c0a5780 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb99c1360e0 con 0x7fb99c0a5780 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb984001510 con 0x7fb99c0a5780 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient(hunting): dump: 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.167 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.168 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.169 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4198250560 0 0) 0x7fb97c003520 con 0x7fb99c13e310 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.81:6789/0 conn(0x7fb99c0a5780 legacy=0x7fb99c1d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.7:6789/0 conn(0x7fb99c0a53a0 legacy=0x7fb99c131b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 monclient: found mon.e 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99c0a1e00 con 0x7fb99c13e310 2024-05-08T15:49:19.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9a1006640 5 monclient: authenticate success, global_id 94238 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9a1006640 10 monclient: _renew_subs 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9a1006640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb99c136d80 con 0x7fb99c13e310 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9a1006640 10 monclient: _renew_subs 2024-05-08T15:49:19.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9a1006640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fb99c1380a0 con 0x7fb99c13e310 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb98c0047e0 con 0x7fb99c13e310 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb98c004cd0 con 0x7fb99c13e310 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb99959a640 4 set_mon_vals no callback set 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.148+0000 7fb9937fe640 10 monclient: dump: 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.172 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fb98c010050 con 0x7fb99c13e310 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9a1006640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb9800065c0 con 0x7fb980002860 2024-05-08T15:49:19.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(100..100 src has 1..100) ==== 3898+0+0 (unknown 251748867 0 0) 0x7fb98c046c80 con 0x7fb99c13e310 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb99a59c640 10 monclient: get_auth_request con 0x7fb97c03a3d0 auth_method 0 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3837379077 0 0) 0x7fb9800065c0 con 0x7fb980002860 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 10 monclient(hunting): my global_id is 75153 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.149+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb97c003520 con 0x7fb980002860 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2810897111 0 0) 0x7fb97c003520 con 0x7fb980002860 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb9800065c0 con 0x7fb980002860 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb994001d80 con 0x7fb980002860 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:19.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 10 monclient: dump: 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.175 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1945567926 0 0) 0x7fb9800065c0 con 0x7fb980002860 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb98000a150 con 0x7fb980002860 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb994005880 con 0x7fb980002860 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.81:6790/0 conn(0x7fb980002860 legacy=0x7fb980004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.150+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.81:6790/0 conn(0x7fb980002860 legacy=0x7fb980004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9a1006640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb980009e40 con 0x7fb980009100 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559862955 0 0) 0x7fb980009e40 con 0x7fb980009100 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9937fe640 10 monclient(hunting): my global_id is 75162 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.157+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9800065c0 con 0x7fb980009100 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1577876967 0 0) 0x7fb9800065c0 con 0x7fb980009100 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb980009e40 con 0x7fb980009100 2024-05-08T15:49:19.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb994006060 con 0x7fb980009100 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 10 monclient: dump: 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.179 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.180 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 805212375 0 0) 0x7fb980009e40 con 0x7fb980009100 2024-05-08T15:49:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb98000c3d0 con 0x7fb980009100 2024-05-08T15:49:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 2086577967) 0x7fb9940083c0 con 0x7fb980009100 2024-05-08T15:49:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.81:6790/0 conn(0x7fb980009100 legacy=0x7fb980009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.158+0000 7fb9937fe640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.81:6790/0 conn(0x7fb980009100 legacy=0x7fb980009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 151448, 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:19.182 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.183 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.184 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:19.185 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.186 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:19.187 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.188 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.189 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.190 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.191 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.192 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.193 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.194 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.195 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.196 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:19.197 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.198 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:19.199 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.200 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.201 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.202 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.203 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:19.204 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.81:6812/847627118 conn(0x7fb97c03a3d0 legacy=0x7fb97c03c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 10 monclient: shutdown 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 >> v1:172.21.15.7:6793/0 conn(0x7fb99c13e310 legacy=0x7fb99c1d4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb99b59e640 1 -- 172.21.15.7:0/568949988 reap_dead start 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb99b59e640 1 -- 172.21.15.7:0/568949988 reap_dead start 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 shutdown_connections 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 >> 172.21.15.7:0/568949988 conn(0x7fb99c0a9fa0 msgr2=0x7fb99c13eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 shutdown_connections 2024-05-08T15:49:19.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.159+0000 7fb9a1006640 1 -- 172.21.15.7:0/568949988 wait complete. 2024-05-08T15:49:19.206 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:19.206 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:49:19.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.313+0000 7fb894bb2640 10 monclient: get_monmap_and_config 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.313+0000 7fb894bb2640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.313+0000 7fb894bb2640 10 monclient: monmap: 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:19.314634+0000 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:19.314634+0000 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.310 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.312 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 1 Processor -- start 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 1 -- start start 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient: init 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient: _add_conns ranks=[2,7,5,4,3,0,1,6,8] 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient(hunting): picked mon.noname-b con 0x7fb890137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient(hunting): picked mon.noname-i con 0x7fb8900a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient(hunting): picked mon.noname-h con 0x7fb8900a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8900a2c70 con 0x7fb890137940 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8900a2f50 con 0x7fb8900a41b0 2024-05-08T15:49:19.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8900a1670 con 0x7fb8900a49a0 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.314+0000 7fb894bb2640 10 monclient(hunting): authenticate will time out at 967.429564s 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88e59c640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fb890137940 0x7fb890137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43674/0 (socket says 172.21.15.7:43674) 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88e59c640 1 -- 172.21.15.7:0/480043780 learned_addr learned my addr 172.21.15.7:0/480043780 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 326522889 0 0) 0x7fb8900a2c70 con 0x7fb890137940 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): my global_id is 94268 2024-05-08T15:49:19.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb87c003520 con 0x7fb890137940 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3311877133 0 0) 0x7fb8900a1670 con 0x7fb8900a49a0 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): my global_id is 94255 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8900a2c70 con 0x7fb8900a49a0 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 992626992 0 0) 0x7fb8900a2f50 con 0x7fb8900a41b0 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient(hunting): my global_id is 5090 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8900a1670 con 0x7fb8900a41b0 2024-05-08T15:49:19.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1835716072 0 0) 0x7fb87c003520 con 0x7fb890137940 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 >> v1:172.21.15.81:6792/0 conn(0x7fb8900a49a0 legacy=0x7fb89013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 >> v1:172.21.15.81:6791/0 conn(0x7fb8900a41b0 legacy=0x7fb890140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: _finish_hunting 0 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 monclient: found mon.noname-b 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8900a39f0 con 0x7fb890137940 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: _finish_auth 0 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: _check_auth_tickets 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb894bb2640 5 monclient: authenticate success, global_id 94268 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb8800017b0 con 0x7fb890137940 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb894bb2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: dump: 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.324 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb894bb2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb880001a10 con 0x7fb890137940 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 1 -- 172.21.15.7:0/480043780 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb880004be0 con 0x7fb890137940 2024-05-08T15:49:19.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb894bb2640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb894bb2640 4 set_mon_vals no callback set 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.315+0000 7fb88dd9b640 10 monclient: dump: 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.326 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.316+0000 7fb894bb2640 10 monclient: shutdown 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.316+0000 7fb894bb2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.316+0000 7fb894bb2640 1 -- 172.21.15.7:0/480043780 >> v1:172.21.15.7:6790/0 conn(0x7fb890137940 legacy=0x7fb890137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.316+0000 7fb894bb2640 1 -- 172.21.15.7:0/480043780 shutdown_connections 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.316+0000 7fb894bb2640 1 -- 172.21.15.7:0/480043780 >> 172.21.15.7:0/480043780 conn(0x7fb8900a9fa0 msgr2=0x7fb8900aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.316+0000 7fb894bb2640 1 -- 172.21.15.7:0/480043780 shutdown_connections 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.316+0000 7fb894bb2640 1 -- 172.21.15.7:0/480043780 wait complete. 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient: monmap: 2024-05-08T15:49:19.327 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.328 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 1 Processor -- start 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 1 -- start start 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient: init 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient: _add_conns ranks=[7,8,6,3,2,4,5,1,0] 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient(hunting): picked mon.noname-i con 0x7fb8900a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient(hunting): picked mon.noname-e con 0x7fb8900a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient(hunting): picked mon.noname-d con 0x7fb890137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb890135f30 con 0x7fb890137940 2024-05-08T15:49:19.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8901340e0 con 0x7fb8900a41b0 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8900a1f30 con 0x7fb8900a49a0 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.317+0000 7fb894bb2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 10 monclient(hunting): authenticate will time out at 967.432613s 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb887fff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fb8900a49a0 0x7fb890132680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:57864/0 (socket says 172.21.15.7:57864) 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb887fff640 1 -- 172.21.15.7:0/1070667554 learned_addr learned my addr 172.21.15.7:0/1070667554 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2904735517 0 0) 0x7fb8900a1f30 con 0x7fb8900a49a0 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): my global_id is 94247 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb870003520 con 0x7fb8900a49a0 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 108553709 0 0) 0x7fb890135f30 con 0x7fb890137940 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): my global_id is 75138 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8900a1f30 con 0x7fb890137940 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2842617104 0 0) 0x7fb8901340e0 con 0x7fb8900a41b0 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): my global_id is 94264 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb890135f30 con 0x7fb8900a41b0 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1374119107 0 0) 0x7fb8900a1f30 con 0x7fb890137940 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb8901340e0 con 0x7fb890137940 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 905478267 0 0) 0x7fb870003520 con 0x7fb8900a49a0 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb8900a1f30 con 0x7fb8900a49a0 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb888001dd0 con 0x7fb890137940 2024-05-08T15:49:19.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.332 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3937420994 0 0) 0x7fb890135f30 con 0x7fb8900a41b0 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb870003520 con 0x7fb8900a41b0 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb878001a20 con 0x7fb8900a49a0 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.333 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb880004b20 con 0x7fb8900a41b0 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:19.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.335 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3799830615 0 0) 0x7fb8901340e0 con 0x7fb890137940 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.7:6793/0 conn(0x7fb8900a49a0 legacy=0x7fb890132680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.81:6792/0 conn(0x7fb8900a41b0 legacy=0x7fb890131f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 monclient: found mon.d 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8900a0f60 con 0x7fb890137940 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb886ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 5 monclient: authenticate success, global_id 75138 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 10 monclient: _renew_subs 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:19.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb890133c40 con 0x7fb890137940 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 10 monclient: _renew_subs 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.318+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fb890135c50 con 0x7fb890137940 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb888003c40 con 0x7fb890137940 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb888004130 con 0x7fb890137940 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb88d59a640 4 set_mon_vals no callback set 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 10 monclient: dump: 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.337 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb894bb2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb8740065c0 con 0x7fb874002860 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fb88800f4b0 con 0x7fb890137940 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.319+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(100..100 src has 1..100) ==== 3898+0+0 (unknown 251748867 0 0) 0x7fb888045ec0 con 0x7fb890137940 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb887fff640 10 monclient: get_auth_request con 0x7fb87003a3a0 auth_method 0 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2960641042 0 0) 0x7fb8740065c0 con 0x7fb874002860 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 10 monclient(hunting): my global_id is 5099 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8901340e0 con 0x7fb874002860 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1181716738 0 0) 0x7fb8901340e0 con 0x7fb874002860 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb8740065c0 con 0x7fb874002860 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb8800016d0 con 0x7fb874002860 2024-05-08T15:49:19.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 10 monclient: dump: 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.340 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 635335140 0 0) 0x7fb8740065c0 con 0x7fb874002860 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.320+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb87400a150 con 0x7fb874002860 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.321+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb880005190 con 0x7fb874002860 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.321+0000 7fb886ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.321+0000 7fb886ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.321+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.81:6791/0 conn(0x7fb874002860 legacy=0x7fb874004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.321+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.81:6791/0 conn(0x7fb874002860 legacy=0x7fb874004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.327+0000 7fb894bb2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.327+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb874009e20 con 0x7fb8740090e0 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1529557639 0 0) 0x7fb874009e20 con 0x7fb8740090e0 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient(hunting): my global_id is 5108 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8740065c0 con 0x7fb8740090e0 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 727881296 0 0) 0x7fb8740065c0 con 0x7fb8740090e0 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb874009e20 con 0x7fb8740090e0 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb880005a00 con 0x7fb8740090e0 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:19.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient: dump: 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.345 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3184130854 0 0) 0x7fb874009e20 con 0x7fb8740090e0 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb87400c390 con 0x7fb8740090e0 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 3092581350) 0x7fb880008640 con 0x7fb8740090e0 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.81:6791/0 conn(0x7fb8740090e0 legacy=0x7fb874009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.328+0000 7fb886ffd640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.81:6791/0 conn(0x7fb8740090e0 legacy=0x7fb874009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:19.346 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 151634, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.347 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.348 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:19.350 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.351 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.352 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.353 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.354 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.355 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.357 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.358 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.359 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.360 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.361 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:19.362 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:19.363 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:19.364 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.81:6812/847627118 conn(0x7fb87003a3a0 legacy=0x7fb87003c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 10 monclient: shutdown 2024-05-08T15:49:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 >> v1:172.21.15.7:6792/0 conn(0x7fb890137940 legacy=0x7fb8901d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb88ed9d640 1 -- 172.21.15.7:0/1070667554 reap_dead start 2024-05-08T15:49:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb88ed9d640 1 -- 172.21.15.7:0/1070667554 reap_dead start 2024-05-08T15:49:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 shutdown_connections 2024-05-08T15:49:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 >> 172.21.15.7:0/1070667554 conn(0x7fb8900a9fa0 msgr2=0x7fb890130a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 shutdown_connections 2024-05-08T15:49:19.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.329+0000 7fb894bb2640 1 -- 172.21.15.7:0/1070667554 wait complete. 2024-05-08T15:49:19.367 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:19.367 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:49:19.367 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:49:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.476+0000 7f1836af2640 10 monclient: get_monmap_and_config 2024-05-08T15:49:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.476+0000 7f1836af2640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.476+0000 7f1836af2640 10 monclient: monmap: 2024-05-08T15:49:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:19.477422+0000 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:19.477422+0000 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.484 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 1 Processor -- start 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 1 -- start start 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient: init 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient: _add_conns ranks=[2,4,0,7,8,5,1,3,6] 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient(hunting): picked mon.noname-b con 0x7f1830137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient(hunting): picked mon.noname-c con 0x7f18300a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient(hunting): picked mon.noname-a con 0x7f18300a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18300a2c70 con 0x7f18300a41b0 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18300a2f50 con 0x7f1830137940 2024-05-08T15:49:19.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18300a1670 con 0x7f18300a49a0 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f1836af2640 10 monclient(hunting): authenticate will time out at 967.592344s 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f182ffff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f18300a49a0 0x7f183013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:58004/0 (socket says 172.21.15.7:58004) 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f183508f640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f18300a41b0 0x7f1830140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:48924/0 (socket says 172.21.15.7:48924) 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f182ffff640 1 -- 172.21.15.7:0/1301193302 learned_addr learned my addr 172.21.15.7:0/1301193302 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.477+0000 7f183488e640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f1830137940 0x7f1830137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43676/0 (socket says 172.21.15.7:43676) 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1273510982 0 0) 0x7f18300a2f50 con 0x7f1830137940 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): my global_id is 94277 2024-05-08T15:49:19.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1820003520 con 0x7f1830137940 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1506664975 0 0) 0x7f18300a2c70 con 0x7f18300a41b0 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): my global_id is 94266 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18300a2f50 con 0x7f18300a41b0 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297447790 0 0) 0x7f18300a1670 con 0x7f18300a49a0 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient(hunting): my global_id is 5134 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18300a2c70 con 0x7f18300a49a0 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3586986103 0 0) 0x7f1820003520 con 0x7f1830137940 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 >> v1:172.21.15.7:6791/0 conn(0x7f18300a49a0 legacy=0x7f183013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 >> v1:172.21.15.7:6789/0 conn(0x7f18300a41b0 legacy=0x7f1830140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 monclient: found mon.noname-b 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18300a39f0 con 0x7f1830137940 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:19.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f18240016e0 con 0x7f1830137940 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 5 monclient: authenticate success, global_id 94277 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: dump: 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.489 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1824001980 con 0x7f1830137940 2024-05-08T15:49:19.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 1 -- 172.21.15.7:0/1301193302 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1824004b20 con 0x7f1830137940 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 4 set_mon_vals no callback set 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f182f7fe640 10 monclient: dump: 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.491 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 10 monclient: shutdown 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.478+0000 7f1836af2640 1 -- 172.21.15.7:0/1301193302 >> v1:172.21.15.7:6790/0 conn(0x7f1830137940 legacy=0x7f1830137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.479+0000 7f1836af2640 1 -- 172.21.15.7:0/1301193302 shutdown_connections 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.479+0000 7f1836af2640 1 -- 172.21.15.7:0/1301193302 >> 172.21.15.7:0/1301193302 conn(0x7f18300a9fa0 msgr2=0x7f18300aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.479+0000 7f1836af2640 1 -- 172.21.15.7:0/1301193302 shutdown_connections 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.479+0000 7f1836af2640 1 -- 172.21.15.7:0/1301193302 wait complete. 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.479+0000 7f1836af2640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.479+0000 7f1836af2640 10 monclient: monmap: 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.493 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.479+0000 7f1836af2640 1 Processor -- start 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 1 -- start start 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient: init 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient: _add_conns ranks=[1,8,5,2,7,6,4,0,3] 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient(hunting): picked mon.noname-f con 0x7f18300a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:19.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient(hunting): picked mon.noname-e con 0x7f18300a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient(hunting): picked mon.noname-h con 0x7f1830137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1830139210 con 0x7f18300a41b0 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18300a0f60 con 0x7f1830137940 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1830136690 con 0x7f18300a49a0 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f1836af2640 10 monclient(hunting): authenticate will time out at 967.595274s 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182ffff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f18300a49a0 0x7f18301cc250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:57872/0 (socket says 172.21.15.7:57872) 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182ffff640 1 -- 172.21.15.7:0/3569326644 learned_addr learned my addr 172.21.15.7:0/3569326644 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880168013 0 0) 0x7f1830136690 con 0x7f18300a49a0 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182d7fa640 10 monclient(hunting): my global_id is 94256 2024-05-08T15:49:19.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.480+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1814003520 con 0x7f18300a49a0 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2166021090 0 0) 0x7f1814003520 con 0x7f18300a49a0 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1830136690 con 0x7f18300a49a0 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f181c001390 con 0x7f18300a49a0 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): dump: 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.496 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3836689302 0 0) 0x7f18300a39f0 con 0x7f18300a41b0 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): my global_id is 5086 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1814003520 con 0x7f18300a41b0 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 239398492 0 0) 0x7f18300a0f60 con 0x7f1830137940 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient(hunting): my global_id is 5117 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18300a39f0 con 0x7f1830137940 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1345891292 0 0) 0x7f1830136690 con 0x7f18300a49a0 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.81:6791/0 conn(0x7f1830137940 legacy=0x7f18301d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.81:6789/0 conn(0x7f18300a41b0 legacy=0x7f1830141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 monclient: found mon.e 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1830141750 con 0x7f18300a49a0 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: _finish_auth 0 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f1836af2640 5 monclient: authenticate success, global_id 94256 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f1836af2640 10 monclient: _renew_subs 2024-05-08T15:49:19.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f1836af2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f18301cc960 con 0x7f18300a49a0 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f1836af2640 10 monclient: _renew_subs 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f1836af2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f18301d5cd0 con 0x7f18300a49a0 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f181c004a90 con 0x7f18300a49a0 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f181c004f80 con 0x7f18300a49a0 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182effd640 4 set_mon_vals no callback set 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.481+0000 7f182d7fa640 10 monclient: dump: 2024-05-08T15:49:19.500 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.501 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f1836af2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f18100065c0 con 0x7f1810002860 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f181c010300 con 0x7f18300a49a0 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(100..100 src has 1..100) ==== 3898+0+0 (unknown 251748867 0 0) 0x7f181c046ea0 con 0x7f18300a49a0 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f183508f640 10 monclient: get_auth_request con 0x7f1814039d40 auth_method 0 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3168282497 0 0) 0x7f18100065c0 con 0x7f1810002860 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 10 monclient(hunting): my global_id is 94273 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.482+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1830136690 con 0x7f1810002860 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1989040609 0 0) 0x7f1830136690 con 0x7f1810002860 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f18100065c0 con 0x7f1810002860 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f18240016a0 con 0x7f1810002860 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 10 monclient: dump: 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.503 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3436843713 0 0) 0x7f18100065c0 con 0x7f1810002860 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f181000a150 con 0x7f1810002860 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f1824004b30 con 0x7f1810002860 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:19.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:19.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.81:6792/0 conn(0x7f1810002860 legacy=0x7f1810004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.483+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.81:6792/0 conn(0x7f1810002860 legacy=0x7f1810004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f1836af2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f1810009e40 con 0x7f1810009100 2024-05-08T15:49:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1551660040 0 0) 0x7f1810009e40 con 0x7f1810009100 2024-05-08T15:49:19.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 10 monclient(hunting): my global_id is 94282 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18100065c0 con 0x7f1810009100 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3962043122 0 0) 0x7f18100065c0 con 0x7f1810009100 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1810009e40 con 0x7f1810009100 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1824001c00 con 0x7f1810009100 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 10 monclient: dump: 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.507 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 445720694 0 0) 0x7f1810009e40 con 0x7f1810009100 2024-05-08T15:49:19.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.490+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f181000c3d0 con 0x7f1810009100 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7522 (unknown 0 0 1216356195) 0x7f1824007440 con 0x7f1810009100 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f182d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f182d7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.81:6792/0 conn(0x7f1810009100 legacy=0x7f1810009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f182d7fa640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.81:6792/0 conn(0x7f1810009100 legacy=0x7f1810009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:19.509 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 20, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 10053, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:49:19.510 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.511 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.512 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:19.513 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:19.514 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:19.515 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.516 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.517 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.518 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:19.519 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.520 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.521 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.522 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.523 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.524 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.525 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.526 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.527 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.528 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:19.529 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:19.530 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.531 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:19.532 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:19.533 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.81:6812/847627118 conn(0x7f1814039d40 legacy=0x7f181403c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f1836af2640 10 monclient: shutdown 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f1836af2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 >> v1:172.21.15.7:6793/0 conn(0x7f18300a49a0 legacy=0x7f18301cc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f183508f640 1 -- 172.21.15.7:0/3569326644 reap_dead start 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.491+0000 7f183508f640 1 -- 172.21.15.7:0/3569326644 reap_dead start 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.492+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 shutdown_connections 2024-05-08T15:49:19.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.492+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 >> 172.21.15.7:0/3569326644 conn(0x7f18300a9fa0 msgr2=0x7f18300a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.492+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 shutdown_connections 2024-05-08T15:49:19.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.492+0000 7f1836af2640 1 -- 172.21.15.7:0/3569326644 wait complete. 2024-05-08T15:49:19.535 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:49:19.535 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['i', 'c'] 2024-05-08T15:49:19.535 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-05-08T15:49:19.535 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.i 2024-05-08T15:49:19.535 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-05-08T15:49:19.536 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-05-08T15:49:19.536 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:49:19.548 INFO:tasks.ceph.mon.i:Stopped 2024-05-08T15:49:19.548 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-05-08T15:49:19.548 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T15:49:19.548 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T15:49:19.549 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:49:19.560 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T15:49:19.560 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-05-08T15:49:19.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:49:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.706+0000 7f6b1b7ce640 10 monclient: get_monmap_and_config 2024-05-08T15:49:19.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.706+0000 7f6b1b7ce640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.706+0000 7f6b1b7ce640 10 monclient: monmap: 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:19.707259+0000 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:19.707259+0000 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.703 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.704 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.706+0000 7f6b1b7ce640 1 Processor -- start 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 1 -- start start 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient: init 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient: _add_conns ranks=[1,6,8,0,5,4,2,7,3] 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient(hunting): picked mon.noname-f con 0x7f6b140a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient(hunting): picked mon.noname-d con 0x7f6b140a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient(hunting): picked mon.noname-e con 0x7f6b1413e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b140a1660 con 0x7f6b140a53a0 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b140a28f0 con 0x7f6b140a5780 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b140a2bd0 con 0x7f6b1413e310 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b1b7ce640 10 monclient(hunting): authenticate will time out at 967.822258s 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b19d6b640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f6b1413e310 0x7f6b14140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:57900/0 (socket says 172.21.15.7:57900) 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b18d69640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f6b140a5780 0x7f6b1413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35632/0 (socket says 172.21.15.7:35632) 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.707+0000 7f6b19d6b640 1 -- 172.21.15.7:0/1892360361 learned_addr learned my addr 172.21.15.7:0/1892360361 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 557106066 0 0) 0x7f6b140a1660 con 0x7f6b140a53a0 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient(hunting): my global_id is 5095 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b00003520 con 0x7f6b140a53a0 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2522356595 0 0) 0x7f6b00003520 con 0x7f6b140a53a0 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 >> v1:172.21.15.7:6793/0 conn(0x7f6b1413e310 legacy=0x7f6b14140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 >> v1:172.21.15.7:6792/0 conn(0x7f6b140a5780 legacy=0x7f6b1413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: _finish_hunting 0 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 monclient: found mon.noname-f 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b140a6730 con 0x7f6b140a53a0 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: _finish_auth 0 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: _check_auth_tickets 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6b100017b0 con 0x7f6b140a53a0 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b1b7ce640 5 monclient: authenticate success, global_id 5095 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b1b7ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: dump: 2024-05-08T15:49:19.710 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.711 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b1b7ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6b10001b10 con 0x7f6b140a53a0 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b1b7ce640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 1 -- 172.21.15.7:0/1892360361 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6b10001d40 con 0x7f6b140a53a0 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b1b7ce640 4 set_mon_vals no callback set 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.708+0000 7f6b0bfff640 10 monclient: dump: 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:19.712 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 10 monclient: shutdown 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1892360361 >> v1:172.21.15.81:6789/0 conn(0x7f6b140a53a0 legacy=0x7f6b140a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:19.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1892360361 shutdown_connections 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1892360361 >> 172.21.15.7:0/1892360361 conn(0x7f6b140a9fa0 msgr2=0x7f6b140aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1892360361 shutdown_connections 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1892360361 wait complete. 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 10 monclient: build_initial_monmap 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.709+0000 7f6b1b7ce640 10 monclient: monmap: 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:19.714 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 1 Processor -- start 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 1 -- start start 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient: init 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient: _add_conns ranks=[8,0,6,5,4,7,2,3,1] 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient(hunting): picked mon.noname-e con 0x7f6b140a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient(hunting): picked mon.noname-a con 0x7f6b140a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient(hunting): picked mon.noname-d con 0x7f6b1413e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:19.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b14135160 con 0x7f6b140a5780 2024-05-08T15:49:19.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b140a43c0 con 0x7f6b1413e310 2024-05-08T15:49:19.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b14134d40 con 0x7f6b140a53a0 2024-05-08T15:49:19.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:19.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1b7ce640 10 monclient(hunting): authenticate will time out at 967.825200s 2024-05-08T15:49:19.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1956a640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f6b140a53a0 0x7f6b1413bc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:57914/0 (socket says 172.21.15.7:57914) 2024-05-08T15:49:19.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:19.710+0000 7f6b1956a640 1 -- 172.21.15.7:0/1656362195 learned_addr learned my addr 172.21.15.7:0/1656362195 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient(hunting): tick 2024-05-08T15:49:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:49:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 1 monclient(hunting): continuing hunt 2024-05-08T15:49:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:49:22.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.7:6793/0 conn(0x7f6b140a53a0 legacy=0x7f6b1413bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.7:6792/0 conn(0x7f6b1413e310 legacy=0x7f6b141d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.7:6789/0 conn(0x7f6b140a5780 legacy=0x7f6b1413c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient: _add_conns ranks=[1,5,4,7,2,3] 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient(hunting): picked mon.noname-f con 0x7f6afc0037e0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient(hunting): picked mon.noname-h con 0x7f6afc007470 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient(hunting): picked mon.noname-c con 0x7f6afc00b1a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b140a1e00 con 0x7f6afc0037e0 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6afc00ffd0 con 0x7f6afc00b1a0 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6afc011260 con 0x7f6afc007470 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b08ff9640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.710+0000 7f6b19d6b640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.7:6791/0 conn(0x7f6afc00b1a0 legacy=0x7f6afc00d660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 281525670 0 0) 0x7f6b140a1e00 con 0x7f6afc0037e0 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:49:22.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 10 monclient(hunting): my global_id is 5104 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6af8003520 con 0x7f6afc0037e0 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1290564132 0 0) 0x7f6af8003520 con 0x7f6afc0037e0 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6b140a1e00 con 0x7f6afc0037e0 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6b10001d40 con 0x7f6afc0037e0 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.711+0000 7f6b09ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient(hunting): dump: 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:22.711 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 607758481 0 0) 0x7f6b140a1e00 con 0x7f6afc0037e0 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.81:6791/0 conn(0x7f6afc007470 legacy=0x7f6afc0098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:22.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.7:6791/0 conn(0x7f6afc00b1a0 legacy=0x7f6afc00d660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 1 monclient: found mon.f 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b19d6b640 1 -- 172.21.15.7:0/1656362195 reap_dead start 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6afc0124f0 con 0x7f6afc0037e0 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: _finish_auth 0 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b1b7ce640 5 monclient: authenticate success, global_id 5104 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b1b7ce640 10 monclient: _renew_subs 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b1b7ce640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6b14134a60 con 0x7f6afc0037e0 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6b10003d70 con 0x7f6afc0037e0 2024-05-08T15:49:22.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6b100042a0 con 0x7f6afc0037e0 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b09ffb640 10 monclient: dump: 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:22.714 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b1b7ce640 10 monclient: _renew_subs 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b0b7fe640 4 set_mon_vals no callback set 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b1b7ce640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.712+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f6b141386a0 con 0x7f6afc0037e0 2024-05-08T15:49:22.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.713+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6b10004500 con 0x7f6afc0037e0 2024-05-08T15:49:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.713+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(100..100 src has 1..100) ==== 3898+0+0 (unknown 251748867 0 0) 0x7f6b10046f60 con 0x7f6afc0037e0 2024-05-08T15:49:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.714+0000 7f6b18d69640 10 monclient: get_auth_request con 0x7f6af8039a50 auth_method 0 2024-05-08T15:49:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.714+0000 7f6b1b7ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.714+0000 7f6b1b7ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:22.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.714+0000 7f6b1b7ce640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:49:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.714+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14132bd0 con 0x7f6afc0037e0 2024-05-08T15:49:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.719+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f6b10016030 con 0x7f6afc0037e0 2024-05-08T15:49:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.719+0000 7f6b09ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:22.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.719+0000 7f6b09ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:22.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.981+0000 7f6b1b7ce640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:49:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.981+0000 7f6b1b7ce640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.981+0000 7f6b1b7ce640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:49:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.981+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6b14075820 con 0x7f6afc0037e0 2024-05-08T15:49:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.982+0000 7f6b09ffb640 1 -- 172.21.15.7:0/1656362195 <== mon.1 v1:172.21.15.81:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 3440748585) 0x7f6b10015930 con 0x7f6afc0037e0 2024-05-08T15:49:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.982+0000 7f6b09ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:22.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.982+0000 7f6b09ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:22.978 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":20,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.81:6812/847627118 conn(0x7f6af8039a50 legacy=0x7f6af803bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 10 monclient: shutdown 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 >> v1:172.21.15.81:6789/0 conn(0x7f6afc0037e0 legacy=0x7f6afc005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 shutdown_connections 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 >> 172.21.15.7:0/1656362195 conn(0x7f6b140a9fa0 msgr2=0x7f6b140a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 shutdown_connections 2024-05-08T15:49:22.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:22.984+0000 7f6b1b7ce640 1 -- 172.21.15.7:0/1656362195 wait complete. 2024-05-08T15:49:25.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.111+0000 7fadcbdbd640 10 monclient: get_monmap_and_config 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.111+0000 7fadcbdbd640 10 monclient: build_initial_monmap 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.111+0000 7fadcbdbd640 10 monclient: monmap: 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:26.112010+0000 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:26.112010+0000 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:26.107 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:26.108 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:26.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.111+0000 7fadcbdbd640 1 Processor -- start 2024-05-08T15:49:26.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.111+0000 7fadcbdbd640 1 -- start start 2024-05-08T15:49:26.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.111+0000 7fadcbdbd640 10 monclient: init 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 10 monclient: _add_conns ranks=[8,0,4,5,2,7,6,3,1] 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 10 monclient(hunting): picked mon.noname-e con 0x7fadc4137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 10 monclient(hunting): picked mon.noname-a con 0x7fadc40a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 10 monclient(hunting): picked mon.noname-c con 0x7fadc40a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40a2c70 con 0x7fadc40a49a0 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40a2f50 con 0x7fadc40a41b0 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40a1670 con 0x7fadc4137940 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadcbdbd640 10 monclient(hunting): authenticate will time out at 974.227107s 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadca35a640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7fadc40a41b0 legacy=0x7fadc4140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadc9358640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fadc40a49a0 0x7fadc413cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47590/0 (socket says 172.21.15.7:47590) 2024-05-08T15:49:26.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadc9358640 1 -- 172.21.15.7:0/1520577683 learned_addr learned my addr 172.21.15.7:0/1520577683 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:26.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.112+0000 7fadc9b59640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fadc4137940 0x7fadc4137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:52728/0 (socket says 172.21.15.7:52728) 2024-05-08T15:49:26.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.313+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6791/0 conn(0x7fadc40a41b0 legacy=0x7fadc4140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:26.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:26.714+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6791/0 conn(0x7fadc40a41b0 legacy=0x7fadc4140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:27.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:27.515+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6791/0 conn(0x7fadc40a41b0 legacy=0x7fadc4140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient(hunting): tick 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 1 monclient(hunting): continuing hunt 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6793/0 conn(0x7fadc4137940 legacy=0x7fadc4137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6791/0 conn(0x7fadc40a41b0 legacy=0x7fadc4140610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6789/0 conn(0x7fadc40a49a0 legacy=0x7fadc413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient: _add_conns ranks=[3,5,6,1,2,7] 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient(hunting): picked mon.noname-g con 0x7fada80037c0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient(hunting): picked mon.noname-h con 0x7fada8007450 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient(hunting): picked mon.noname-d con 0x7fada800b1e0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40a39f0 con 0x7fada80037c0 2024-05-08T15:49:29.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40a2f50 con 0x7fada8007450 2024-05-08T15:49:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada8011000 con 0x7fada800b1e0 2024-05-08T15:49:29.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:29.112+0000 7fadbb7fe640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:32.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.112+0000 7fadbb7fe640 10 monclient(hunting): tick 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.112+0000 7fadbb7fe640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.112+0000 7fadbb7fe640 1 monclient(hunting): continuing hunt 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.112+0000 7fadbb7fe640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6792/0 conn(0x7fada800b1e0 legacy=0x7fada800d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6791/0 conn(0x7fada8007450 legacy=0x7fada8009910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.112+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6790/0 conn(0x7fada80037c0 legacy=0x7fada8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 10 monclient: _add_conns ranks=[1,2,7] 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 reap_dead start 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 10 monclient(hunting): picked mon.noname-f con 0x7fada8010160 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 10 monclient(hunting): picked mon.noname-b con 0x7fada800de50 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 10 monclient(hunting): picked mon.noname-i con 0x7fada800e230 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:32.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada8012290 con 0x7fada8010160 2024-05-08T15:49:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada800e6f0 con 0x7fada800de50 2024-05-08T15:49:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fada800e970 con 0x7fada800e230 2024-05-08T15:49:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadbb7fe640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:32.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.113+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6792/0 conn(0x7fada800e230 legacy=0x7fada80098f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:49:32.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.314+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6792/0 conn(0x7fada800e230 legacy=0x7fada80098f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:49:32.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:32.715+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6792/0 conn(0x7fada800e230 legacy=0x7fada80098f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:49:33.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.516+0000 7fadca35a640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6792/0 conn(0x7fada800e230 legacy=0x7fada80098f0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:49:33.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3580012600 0 0) 0x7fada8012290 con 0x7fada8010160 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): my global_id is 5113 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadb0003520 con 0x7fada8010160 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4011942821 0 0) 0x7fada800e6f0 con 0x7fada800de50 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 10 monclient(hunting): my global_id is 94286 2024-05-08T15:49:33.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.572+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fada8012290 con 0x7fada800de50 2024-05-08T15:49:33.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3080914609 0 0) 0x7fadb0003520 con 0x7fada8010160 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6792/0 conn(0x7fada800e230 legacy=0x7fada80098f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.7:6790/0 conn(0x7fada800de50 legacy=0x7fada8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 10 monclient: _finish_hunting 0 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 1 monclient: found mon.noname-f 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada800ebf0 con 0x7fada8010160 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 10 monclient: _finish_auth 0 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 10 monclient: _check_auth_tickets 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fadc00016b0 con 0x7fada8010160 2024-05-08T15:49:33.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadcbdbd640 5 monclient: authenticate success, global_id 5113 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadcbdbd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadc8b57640 10 monclient: dump: 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.574 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.576+0000 7fadcbdbd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.580+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fadc0001980 con 0x7fada8010160 2024-05-08T15:49:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.581+0000 7fadc8b57640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.581+0000 7fadc8b57640 1 -- 172.21.15.7:0/1520577683 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fadc0004b20 con 0x7fada8010160 2024-05-08T15:49:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.581+0000 7fadcbdbd640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.581+0000 7fadcbdbd640 4 set_mon_vals no callback set 2024-05-08T15:49:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadcbdbd640 10 monclient: shutdown 2024-05-08T15:49:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadc8b57640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadc8b57640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadc8b57640 10 monclient: dump: 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadcbdbd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadcbdbd640 1 -- 172.21.15.7:0/1520577683 >> v1:172.21.15.81:6789/0 conn(0x7fada8010160 legacy=0x7fada80105a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadcbdbd640 1 -- 172.21.15.7:0/1520577683 shutdown_connections 2024-05-08T15:49:33.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.582+0000 7fadcbdbd640 1 -- 172.21.15.7:0/1520577683 >> 172.21.15.7:0/1520577683 conn(0x7fadc40a9fa0 msgr2=0x7fadc40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:33.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.583+0000 7fadcbdbd640 1 -- 172.21.15.7:0/1520577683 shutdown_connections 2024-05-08T15:49:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.583+0000 7fadcbdbd640 1 -- 172.21.15.7:0/1520577683 wait complete. 2024-05-08T15:49:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.584+0000 7fadcbdbd640 10 monclient: build_initial_monmap 2024-05-08T15:49:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.584+0000 7fadcbdbd640 10 monclient: monmap: 2024-05-08T15:49:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:33.583 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.585+0000 7fadcbdbd640 1 Processor -- start 2024-05-08T15:49:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.585+0000 7fadcbdbd640 1 -- start start 2024-05-08T15:49:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.585+0000 7fadcbdbd640 10 monclient: init 2024-05-08T15:49:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 10 monclient: _add_conns ranks=[0,3,4,2,5,1,8,7,6] 2024-05-08T15:49:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 10 monclient(hunting): picked mon.noname-a con 0x7fada800de50 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 10 monclient(hunting): picked mon.noname-g con 0x7fada800e230 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:33.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 10 monclient(hunting): picked mon.noname-c con 0x7fada8010160 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc4072440 con 0x7fada800de50 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40a1f30 con 0x7fada800e230 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc40a0f60 con 0x7fada8010160 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadcbdbd640 10 monclient(hunting): authenticate will time out at 981.700886s 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadca35a640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7fada8010160 legacy=0x7fadc41d9680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadc9b59640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fada800de50 0x7fadc4133be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47598/0 (socket says 172.21.15.7:47598) 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.586+0000 7fadc9b59640 1 -- 172.21.15.7:0/3333238589 learned_addr learned my addr 172.21.15.7:0/3333238589 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1064262577 0 0) 0x7fadc4072440 con 0x7fada800de50 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): my global_id is 94275 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fada800e930 con 0x7fada800de50 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 129996222 0 0) 0x7fadc40a1f30 con 0x7fada800e230 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 10 monclient(hunting): my global_id is 75171 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.587+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fadc4072440 con 0x7fada800e230 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2648390811 0 0) 0x7fadc4072440 con 0x7fada800e230 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadc40a1f30 con 0x7fada800e230 2024-05-08T15:49:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fadb40019d0 con 0x7fada800e230 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient(hunting): dump: 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 83574537 0 0) 0x7fada800e930 con 0x7fada800de50 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fadc4072440 con 0x7fada800de50 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fadc0001670 con 0x7fada800de50 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:33.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient(hunting): dump: 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.594 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 220860877 0 0) 0x7fadc40a1f30 con 0x7fada800e230 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 >> v1:172.21.15.7:6791/0 conn(0x7fada8010160 legacy=0x7fadc41d9680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 >> v1:172.21.15.7:6789/0 conn(0x7fada800de50 legacy=0x7fadc4133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 monclient: found mon.g 2024-05-08T15:49:33.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:33.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:33.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc4072160 con 0x7fada800e230 2024-05-08T15:49:33.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.588+0000 7fadba7fc640 10 monclient: _finish_auth 0 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadcbdbd640 5 monclient: authenticate success, global_id 75171 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadcbdbd640 10 monclient: _renew_subs 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadcbdbd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fadc40709c0 con 0x7fada800e230 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fadb40047e0 con 0x7fada800e230 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadbbfff640 4 set_mon_vals no callback set 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fadb4004d50 con 0x7fada800e230 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:33.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadba7fc640 10 monclient: dump: 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.598 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadcbdbd640 10 monclient: _renew_subs 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadcbdbd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.589+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fadc41344d0 con 0x7fada800e230 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.590+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fadb40100d0 con 0x7fada800e230 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.590+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(101..101 src has 1..101) ==== 3898+0+0 (unknown 3858878525 0 0) 0x7fadb4046c20 con 0x7fada800e230 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.590+0000 7fadc9b59640 10 monclient: get_auth_request con 0x7fada803c360 auth_method 0 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.590+0000 7fadcbdbd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.590+0000 7fadcbdbd640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:33.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.590+0000 7fadcbdbd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:33.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.590+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadc40a9710 con 0x7fada800e230 2024-05-08T15:49:33.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.596+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fadc40a9710 con 0x7fada800e230 2024-05-08T15:49:33.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.596+0000 7fadba7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:33.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.596+0000 7fadba7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35406]: starting. 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35406]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35308 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35406]: created object 0... 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35406]: created object 25... 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35406]: created object 49... 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35406]: finishing. 2024-05-08T15:49:33.659 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35406]: shutting down. 2024-05-08T15:49:33.669 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:33.669 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35408]: starting. 2024-05-08T15:49:33.669 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35408]: listing objects. 2024-05-08T15:49:33.669 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35408]: listed object 0... 2024-05-08T15:49:33.669 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35408]: listed object 25... 2024-05-08T15:49:33.669 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35408]: saw 50 objects 2024-05-08T15:49:33.669 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35408]: shutting down. 2024-05-08T15:49:33.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.862+0000 7fadcbdbd640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:49:33.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.862+0000 7fadcbdbd640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:33.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.862+0000 7fadcbdbd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:33.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.862+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 --> v1:172.21.15.81:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fadc40753d0 con 0x7fada800e230 2024-05-08T15:49:33.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.862+0000 7fadba7fc640 1 -- 172.21.15.7:0/3333238589 <== mon.3 v1:172.21.15.81:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2690 (unknown 254746657 0 1934368215) 0x7fadb4015c30 con 0x7fada800e230 2024-05-08T15:49:33.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.862+0000 7fadba7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:49:33.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.862+0000 7fadba7fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:33.859 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":24,"quorum":[0,1,2,3,5,6,8],"quorum_names":["a","f","b","g","h","d","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.864+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 >> v1:172.21.15.81:6812/847627118 conn(0x7fada803c360 legacy=0x7fada803e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.864+0000 7fadcbdbd640 10 monclient: shutdown 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.864+0000 7fadcbdbd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.864+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 >> v1:172.21.15.81:6790/0 conn(0x7fada800e230 legacy=0x7fadc41d5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.864+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 shutdown_connections 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.864+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 >> 172.21.15.7:0/3333238589 conn(0x7fadc40a9fa0 msgr2=0x7fadc409abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.865+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 shutdown_connections 2024-05-08T15:49:33.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.865+0000 7fadcbdbd640 1 -- 172.21.15.7:0/3333238589 wait complete. 2024-05-08T15:49:33.871 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-05-08T15:49:33.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-05-08T15:49:33.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.980+0000 7f2cd3f70640 10 monclient: get_monmap_and_config 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.980+0000 7f2cd3f70640 10 monclient: build_initial_monmap 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.980+0000 7f2cd3f70640 10 monclient: monmap: 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:33.981029+0000 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:33.981029+0000 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:33.976 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:33.977 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:33.977 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:33.977 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:33.977 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:33.977 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:33.977 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:33.977 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.980+0000 7f2cd3f70640 1 Processor -- start 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.980+0000 7f2cd3f70640 1 -- start start 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.980+0000 7f2cd3f70640 10 monclient: init 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 10 monclient: _add_conns ranks=[0,8,7,1,3,4,6,5,2] 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 10 monclient(hunting): picked mon.noname-a con 0x7f2ccc0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 10 monclient(hunting): picked mon.noname-e con 0x7f2ccc0a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 10 monclient(hunting): picked mon.noname-i con 0x7f2ccc13e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0a1660 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0a28f0 con 0x7f2ccc13e310 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0a2bd0 con 0x7f2ccc0a5780 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd3f70640 10 monclient(hunting): authenticate will time out at 982.096131s 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd1d0c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2ccc0a53a0 0x7f2ccc0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47608/0 (socket says 172.21.15.7:47608) 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd1d0c640 1 -- 172.21.15.7:0/971424835 learned_addr learned my addr 172.21.15.7:0/971424835 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd150b640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f2ccc0a5780 0x7f2ccc13cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:52734/0 (socket says 172.21.15.7:52734) 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd250d640 1 -- 172.21.15.7:0/971424835 >> v1:172.21.15.81:6792/0 conn(0x7f2ccc13e310 legacy=0x7f2ccc140700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077951020 0 0) 0x7f2ccc0a1660 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): my global_id is 94284 2024-05-08T15:49:33.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb8003520 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499505005 0 0) 0x7f2ccc0a2bd0 con 0x7f2ccc0a5780 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 10 monclient(hunting): my global_id is 94265 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.981+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ccc0a1660 con 0x7f2ccc0a5780 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4139858537 0 0) 0x7f2ccc0a1660 con 0x7f2ccc0a5780 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 >> v1:172.21.15.81:6792/0 conn(0x7f2ccc13e310 legacy=0x7f2ccc140700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 >> v1:172.21.15.7:6789/0 conn(0x7f2ccc0a53a0 legacy=0x7f2ccc0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: _finish_hunting 0 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 monclient: found mon.noname-e 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc0a6730 con 0x7f2ccc0a5780 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: _finish_auth 0 2024-05-08T15:49:33.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: _check_auth_tickets 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2cbc001790 con 0x7f2ccc0a5780 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: dump: 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.989 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 5 monclient: authenticate success, global_id 94265 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2cbc001af0 con 0x7f2ccc0a5780 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 1 -- 172.21.15.7:0/971424835 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2cbc004b20 con 0x7f2ccc0a5780 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd0d0a640 10 monclient: dump: 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.990 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:33.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 4 set_mon_vals no callback set 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 10 monclient: shutdown 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 1 -- 172.21.15.7:0/971424835 >> v1:172.21.15.7:6793/0 conn(0x7f2ccc0a5780 legacy=0x7f2ccc13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 1 -- 172.21.15.7:0/971424835 shutdown_connections 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 1 -- 172.21.15.7:0/971424835 >> 172.21.15.7:0/971424835 conn(0x7f2ccc0a9fa0 msgr2=0x7f2ccc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 1 -- 172.21.15.7:0/971424835 shutdown_connections 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.982+0000 7f2cd3f70640 1 -- 172.21.15.7:0/971424835 wait complete. 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.983+0000 7f2cd3f70640 10 monclient: build_initial_monmap 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.983+0000 7f2cd3f70640 10 monclient: monmap: 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:33.992 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.983+0000 7f2cd3f70640 1 Processor -- start 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.983+0000 7f2cd3f70640 1 -- start start 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.983+0000 7f2cd3f70640 10 monclient: init 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:33.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 10 monclient: _add_conns ranks=[6,1,4,5,3,2,0,7,8] 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 10 monclient(hunting): picked mon.noname-d con 0x7f2ccc0a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 10 monclient(hunting): picked mon.noname-f con 0x7f2ccc0a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 10 monclient(hunting): picked mon.noname-c con 0x7f2ccc13e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc1ccea0 con 0x7f2ccc0a5780 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc1377f0 con 0x7f2ccc13e310 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ccc0a1e00 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd3f70640 10 monclient(hunting): authenticate will time out at 982.098959s 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd250d640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7f2ccc13e310 legacy=0x7f2ccc1d4680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd1d0c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f2ccc0a53a0 0x7f2ccc1cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:34896/0 (socket says 172.21.15.7:34896) 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cd1d0c640 1 -- 172.21.15.7:0/1894893978 learned_addr learned my addr 172.21.15.7:0/1894893978 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2672136862 0 0) 0x7f2ccc0a1e00 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): my global_id is 75147 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb0003520 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002752643 0 0) 0x7f2ccc1ccea0 con 0x7f2ccc0a5780 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): my global_id is 5122 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ccc0a1e00 con 0x7f2ccc0a5780 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 940613904 0 0) 0x7f2cb0003520 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2ccc1ccea0 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2cc8001480 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:33.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): dump: 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.996 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3026624813 0 0) 0x7f2ccc0a1e00 con 0x7f2ccc0a5780 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.984+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cb0003520 con 0x7f2ccc0a5780 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2cbc001750 con 0x7f2ccc0a5780 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient(hunting): dump: 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:33.997 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:33.998 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3106943834 0 0) 0x7f2ccc1ccea0 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.7:6791/0 conn(0x7f2ccc13e310 legacy=0x7f2ccc1d4680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.81:6789/0 conn(0x7f2ccc0a5780 legacy=0x7f2ccc1cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: _finish_hunting 0 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 monclient: found mon.d 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc137230 con 0x7f2ccc0a53a0 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: _finish_auth 0 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: _check_auth_tickets 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 5 monclient: authenticate success, global_id 75147 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 10 monclient: _renew_subs 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:33.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc137510 con 0x7f2ccc0a53a0 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 10 monclient: _renew_subs 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f2ccc135be0 con 0x7f2ccc0a53a0 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2cc8004a90 con 0x7f2ccc0a53a0 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2cc8004f80 con 0x7f2ccc0a53a0 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc3fff640 4 set_mon_vals no callback set 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cc27fc640 10 monclient: dump: 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.000 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.001 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.985+0000 7f2cd3f70640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2cb40065c0 con 0x7f2cb4002860 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2cc8010300 con 0x7f2ccc0a53a0 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(101..101 src has 1..101) ==== 3898+0+0 (unknown 3858878525 0 0) 0x7f2cc8046f60 con 0x7f2ccc0a53a0 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794696061 0 0) 0x7f2cb40065c0 con 0x7f2cb4002860 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 10 monclient(hunting): my global_id is 94293 2024-05-08T15:49:34.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ccc1ccea0 con 0x7f2cb4002860 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cd250d640 10 monclient: get_auth_request con 0x7f2cb0039fa0 auth_method 0 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2931328393 0 0) 0x7f2ccc1ccea0 con 0x7f2cb4002860 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cb40065c0 con 0x7f2cb4002860 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2cbc005170 con 0x7f2cb4002860 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 10 monclient: dump: 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.003 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 502652764 0 0) 0x7f2cb40065c0 con 0x7f2cb4002860 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.986+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2cb400a150 con 0x7f2cb4002860 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.987+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2cbc0052a0 con 0x7f2cb4002860 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.987+0000 7f2cc27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.987+0000 7f2cc27fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:34.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.987+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.7:6789/0 conn(0x7f2cb4002860 legacy=0x7f2cb4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.987+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.7:6789/0 conn(0x7f2cb4002860 legacy=0x7f2cb4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cd3f70640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2cb4009e40 con 0x7f2cb4009100 2024-05-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1305049634 0 0) 0x7f2cb4009e40 con 0x7f2cb4009100 2024-05-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cc27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cc27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cc27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cc27fc640 10 monclient(hunting): my global_id is 94302 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.993+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cb40065c0 con 0x7f2cb4009100 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2157648120 0 0) 0x7f2cb40065c0 con 0x7f2cb4009100 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2cb4009e40 con 0x7f2cb4009100 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2cbc005b70 con 0x7f2cb4009100 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 10 monclient: dump: 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2903511586 0 0) 0x7f2cb4009e40 con 0x7f2cb4009100 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2cb400c3d0 con 0x7f2cb4009100 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7500 (unknown 0 0 537910422) 0x7f2cbc008750 con 0x7f2cb4009100 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.7:6789/0 conn(0x7f2cb4009100 legacy=0x7f2cb4009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cc27fc640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.7:6789/0 conn(0x7f2cb4009100 legacy=0x7f2cb4009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 24, 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 24545, 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:34.009 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.010 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.011 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:34.012 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.013 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:34.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.015 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.022 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.023 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.024 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.025 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.026 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:34.027 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.81:6812/847627118 conn(0x7f2cb0039fa0 legacy=0x7f2cb003c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 10 monclient: shutdown 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 >> v1:172.21.15.7:6792/0 conn(0x7f2ccc0a53a0 legacy=0x7f2ccc1cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd250d640 1 -- 172.21.15.7:0/1894893978 reap_dead start 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd250d640 1 -- 172.21.15.7:0/1894893978 reap_dead start 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 shutdown_connections 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 >> 172.21.15.7:0/1894893978 conn(0x7f2ccc0a9fa0 msgr2=0x7f2ccc1389c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 shutdown_connections 2024-05-08T15:49:34.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:33.994+0000 7f2cd3f70640 1 -- 172.21.15.7:0/1894893978 wait complete. 2024-05-08T15:49:34.030 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:49:34.030 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-05-08T15:49:34.030 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:49:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.138+0000 7f4eb4932640 10 monclient: get_monmap_and_config 2024-05-08T15:49:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.138+0000 7f4eb4932640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.138+0000 7f4eb4932640 10 monclient: monmap: 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:34.139472+0000 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:34.139472+0000 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.135 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.136 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.136 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 1 Processor -- start 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 1 -- start start 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient: init 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient: _add_conns ranks=[8,0,3,1,2,6,5,7,4] 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient(hunting): picked mon.noname-e con 0x7f4eb0137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient(hunting): picked mon.noname-a con 0x7f4eb00a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient(hunting): picked mon.noname-g con 0x7f4eb00a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00a2c70 con 0x7f4eb00a49a0 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00a2f50 con 0x7f4eb00a41b0 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00a1670 con 0x7f4eb0137940 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eb4932640 10 monclient(hunting): authenticate will time out at 982.254407s 2024-05-08T15:49:34.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eadd9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4eb00a49a0 0x7f4eb013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47620/0 (socket says 172.21.15.7:47620) 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eadd9b640 1 -- 172.21.15.7:0/3821781358 learned_addr learned my addr 172.21.15.7:0/3821781358 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.139+0000 7f4eae59c640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f4eb0137940 0x7f4eb0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:52738/0 (socket says 172.21.15.7:52738) 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 923940208 0 0) 0x7f4eb00a1670 con 0x7f4eb0137940 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): my global_id is 94274 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e9c003520 con 0x7f4eb0137940 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 897198291 0 0) 0x7f4eb00a2c70 con 0x7f4eb00a49a0 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): my global_id is 94311 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb00a1670 con 0x7f4eb00a49a0 2024-05-08T15:49:34.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2878543346 0 0) 0x7f4eb00a2f50 con 0x7f4eb00a41b0 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient(hunting): my global_id is 75180 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4eb00a2c70 con 0x7f4eb00a41b0 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2492750126 0 0) 0x7f4e9c003520 con 0x7f4eb0137940 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 >> v1:172.21.15.81:6790/0 conn(0x7f4eb00a41b0 legacy=0x7f4eb0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 >> v1:172.21.15.7:6789/0 conn(0x7f4eb00a49a0 legacy=0x7f4eb013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 monclient: found mon.noname-e 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb00a39f0 con 0x7f4eb0137940 2024-05-08T15:49:34.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4eb4932640 5 monclient: authenticate success, global_id 94274 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4ea00017b0 con 0x7f4eb0137940 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4eb4932640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: dump: 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.147 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4eb4932640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4ea0001ad0 con 0x7f4eb0137940 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 1 -- 172.21.15.7:0/3821781358 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4ea0001d40 con 0x7f4eb0137940 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4eb4932640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4eb4932640 4 set_mon_vals no callback set 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.140+0000 7f4ead59a640 10 monclient: dump: 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.149 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 10 monclient: shutdown 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 1 -- 172.21.15.7:0/3821781358 >> v1:172.21.15.7:6793/0 conn(0x7f4eb0137940 legacy=0x7f4eb0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 1 -- 172.21.15.7:0/3821781358 shutdown_connections 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 1 -- 172.21.15.7:0/3821781358 >> 172.21.15.7:0/3821781358 conn(0x7f4eb00a9fa0 msgr2=0x7f4eb00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 1 -- 172.21.15.7:0/3821781358 shutdown_connections 2024-05-08T15:49:34.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 1 -- 172.21.15.7:0/3821781358 wait complete. 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.141+0000 7f4eb4932640 10 monclient: monmap: 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.151 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 1 Processor -- start 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 1 -- start start 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient: init 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient: _add_conns ranks=[7,0,4,5,8,2,3,6,1] 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient(hunting): picked mon.noname-i con 0x7f4eb00a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient(hunting): picked mon.noname-a con 0x7f4eb00a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient(hunting): picked mon.noname-c con 0x7f4eb0137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00a0f60 con 0x7f4eb00a49a0 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb00a1f30 con 0x7f4eb0137940 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4eb0141840 con 0x7f4eb00a41b0 2024-05-08T15:49:34.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eb4932640 10 monclient(hunting): authenticate will time out at 982.257335s 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eaed9d640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7f4eb0137940 legacy=0x7f4eb0134b90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eae59c640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7f4eb00a41b0 legacy=0x7f4eb0133d70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eadd9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4eb00a49a0 0x7f4eb0134480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47622/0 (socket says 172.21.15.7:47622) 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4eadd9b640 1 -- 172.21.15.7:0/3992925838 learned_addr learned my addr 172.21.15.7:0/3992925838 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3469168174 0 0) 0x7f4e98001a10 con 0x7f4eb00a49a0 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4ea6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4ea6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4ea6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4ea6ffd640 10 monclient(hunting): my global_id is 94320 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.142+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e90003520 con 0x7f4eb00a49a0 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 185433459 0 0) 0x7f4e90003520 con 0x7f4eb00a49a0 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e98001a10 con 0x7f4eb00a49a0 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4e98001ce0 con 0x7f4eb00a49a0 2024-05-08T15:49:34.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.154 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2904176930 0 0) 0x7f4e98001a10 con 0x7f4eb00a49a0 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.81:6792/0 conn(0x7f4eb00a41b0 legacy=0x7f4eb0133d70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.7:6791/0 conn(0x7f4eb0137940 legacy=0x7f4eb0134b90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 monclient: found mon.a 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:34.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb0138df0 con 0x7f4eb00a49a0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eb4932640 5 monclient: authenticate success, global_id 94320 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eb4932640 10 monclient: _renew_subs 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eb4932640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb01365a0 con 0x7f4eb00a49a0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eb4932640 10 monclient: _renew_subs 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eb4932640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4eb01d5e60 con 0x7f4eb00a49a0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4e98004a90 con 0x7f4eb00a49a0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4e98005000 con 0x7f4eb00a49a0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4eacd99640 4 set_mon_vals no callback set 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.143+0000 7f4ea6ffd640 10 monclient: dump: 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.157 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4eb4932640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f4e98005220 con 0x7f4eb00a49a0 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4e940065c0 con 0x7f4e94002860 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(101..101 src has 1..101) ==== 3898+0+0 (unknown 3858878525 0 0) 0x7f4e98047210 con 0x7f4eb00a49a0 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1456534554 0 0) 0x7f4e940065c0 con 0x7f4e94002860 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 10 monclient(hunting): my global_id is 94295 2024-05-08T15:49:34.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e98001a10 con 0x7f4e94002860 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3480251249 0 0) 0x7f4e98001a10 con 0x7f4e94002860 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e940065c0 con 0x7f4e94002860 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4ea0001d40 con 0x7f4e94002860 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4eaed9d640 10 monclient: get_auth_request con 0x7f4e90039a30 auth_method 0 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.144+0000 7f4ea6ffd640 10 monclient: dump: 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.159 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.145+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4156771971 0 0) 0x7f4e940065c0 con 0x7f4e94002860 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.145+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e9400a150 con 0x7f4e94002860 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.145+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4ea00013b0 con 0x7f4e94002860 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.145+0000 7f4ea6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.145+0000 7f4ea6ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.145+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.7:6790/0 conn(0x7f4e94002860 legacy=0x7f4e94004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.145+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.7:6790/0 conn(0x7f4e94002860 legacy=0x7f4e94004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.151+0000 7f4eb4932640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.151+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4e94009e20 con 0x7f4e940090e0 2024-05-08T15:49:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1659743423 0 0) 0x7f4e94009e20 con 0x7f4e940090e0 2024-05-08T15:49:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient(hunting): my global_id is 94304 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e940065c0 con 0x7f4e940090e0 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2974909648 0 0) 0x7f4e940065c0 con 0x7f4e940090e0 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4e94009e20 con 0x7f4e940090e0 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4ea00017d0 con 0x7f4e940090e0 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient: dump: 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.163 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2773139041 0 0) 0x7f4e94009e20 con 0x7f4e940090e0 2024-05-08T15:49:34.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4e9400c390 con 0x7f4e940090e0 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7500 (unknown 0 0 1343719158) 0x7f4ea0007470 con 0x7f4e940090e0 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.7:6790/0 conn(0x7f4e940090e0 legacy=0x7f4e94009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.152+0000 7f4ea6ffd640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.7:6790/0 conn(0x7f4e940090e0 legacy=0x7f4e94009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:34.165 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 24, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 24703, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.166 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.167 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:34.169 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.170 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:34.172 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.175 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.176 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.177 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.178 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:34.179 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.181 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.182 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:34.183 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.81:6812/847627118 conn(0x7f4e90039a30 legacy=0x7f4e9003bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 10 monclient: shutdown 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 >> v1:172.21.15.7:6789/0 conn(0x7f4eb00a49a0 legacy=0x7f4eb0134480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eaed9d640 1 -- 172.21.15.7:0/3992925838 reap_dead start 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eaed9d640 1 -- 172.21.15.7:0/3992925838 reap_dead start 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 shutdown_connections 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 >> 172.21.15.7:0/3992925838 conn(0x7f4eb00a9fa0 msgr2=0x7f4eb01307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 shutdown_connections 2024-05-08T15:49:34.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.153+0000 7f4eb4932640 1 -- 172.21.15.7:0/3992925838 wait complete. 2024-05-08T15:49:34.185 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:49:34.186 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon d 2024-05-08T15:49:34.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.294+0000 7f96c51db640 10 monclient: get_monmap_and_config 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.294+0000 7f96c51db640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.294+0000 7f96c51db640 10 monclient: monmap: 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:34.295023+0000 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:34.295023+0000 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.290 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.291 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.294+0000 7f96c51db640 1 Processor -- start 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.294+0000 7f96c51db640 1 -- start start 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.294+0000 7f96c51db640 10 monclient: init 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 10 monclient: _add_conns ranks=[1,8,0,3,5,6,7,4,2] 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 10 monclient(hunting): picked mon.noname-f con 0x7f96c00a5430 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 10 monclient(hunting): picked mon.noname-e con 0x7f96c00a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:34.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 10 monclient(hunting): picked mon.noname-a con 0x7f96c013e380 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c00a2990 con 0x7f96c013e380 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c00a2c70 con 0x7f96c00a5430 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c00a2f50 con 0x7f96c00a49a0 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96c51db640 10 monclient(hunting): authenticate will time out at 982.409994s 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bf59e640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f96c013e380 0x7f96c0140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:47626/0 (socket says 172.21.15.7:47626) 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96be59c640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f96c00a49a0 0x7f96c013cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:52750/0 (socket says 172.21.15.7:52750) 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bf59e640 1 -- 172.21.15.7:0/585188549 learned_addr learned my addr 172.21.15.7:0/585188549 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1720211286 0 0) 0x7f96c00a2f50 con 0x7f96c00a49a0 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): my global_id is 94283 2024-05-08T15:49:34.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96a8003520 con 0x7f96c00a49a0 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3999622857 0 0) 0x7f96c00a2990 con 0x7f96c013e380 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): my global_id is 94329 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c00a2f50 con 0x7f96c013e380 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753438754 0 0) 0x7f96c00a2c70 con 0x7f96c00a5430 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 10 monclient(hunting): my global_id is 5131 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c00a2990 con 0x7f96c00a5430 2024-05-08T15:49:34.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638820485 0 0) 0x7f96a8003520 con 0x7f96c00a49a0 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 >> v1:172.21.15.81:6789/0 conn(0x7f96c00a5430 legacy=0x7f96c00a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.295+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 >> v1:172.21.15.7:6789/0 conn(0x7f96c013e380 legacy=0x7f96c0140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 1 monclient: found mon.noname-e 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c00a4300 con 0x7f96c00a49a0 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b40017b0 con 0x7f96c00a49a0 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 5 monclient: authenticate success, global_id 94283 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: dump: 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.304 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f96b4001ad0 con 0x7f96c00a49a0 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 1 -- 172.21.15.7:0/585188549 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b4001d40 con 0x7f96c00a49a0 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 4 set_mon_vals no callback set 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96bdd9b640 10 monclient: dump: 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.305 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 10 monclient: shutdown 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 1 -- 172.21.15.7:0/585188549 >> v1:172.21.15.7:6793/0 conn(0x7f96c00a49a0 legacy=0x7f96c013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 1 -- 172.21.15.7:0/585188549 shutdown_connections 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 1 -- 172.21.15.7:0/585188549 >> 172.21.15.7:0/585188549 conn(0x7f96c00a9fa0 msgr2=0x7f96c00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 1 -- 172.21.15.7:0/585188549 shutdown_connections 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.296+0000 7f96c51db640 1 -- 172.21.15.7:0/585188549 wait complete. 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.297+0000 7f96c51db640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.297+0000 7f96c51db640 10 monclient: monmap: 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:34.307 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.297+0000 7f96c51db640 1 Processor -- start 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.297+0000 7f96c51db640 1 -- start start 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.297+0000 7f96c51db640 10 monclient: init 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 10 monclient: _add_conns ranks=[2,5,0,6,1,8,4,3,7] 2024-05-08T15:49:34.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 10 monclient(hunting): picked mon.noname-b con 0x7f96c00a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 10 monclient(hunting): picked mon.noname-h con 0x7f96c00a5430 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 10 monclient(hunting): picked mon.noname-a con 0x7f96c013e380 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c009f9c0 con 0x7f96c013e380 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c00a1f30 con 0x7f96c00a49a0 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96c00a0f60 con 0x7f96c00a5430 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96c51db640 10 monclient(hunting): authenticate will time out at 982.413090s 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96bed9d640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f96c00a49a0 0x7f96c01c9510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:45814/0 (socket says 172.21.15.7:45814) 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96bed9d640 1 -- 172.21.15.7:0/2768363697 learned_addr learned my addr 172.21.15.7:0/2768363697 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1297776179 0 0) 0x7f96c00a1f30 con 0x7f96c00a49a0 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): my global_id is 94313 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96ac003520 con 0x7f96c00a49a0 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2099602407 0 0) 0x7f96c009f9c0 con 0x7f96c013e380 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): my global_id is 94338 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c00a1f30 con 0x7f96c013e380 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651751096 0 0) 0x7f96c00a0f60 con 0x7f96c00a5430 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 10 monclient(hunting): my global_id is 5126 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.298+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c009f9c0 con 0x7f96c00a5430 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2493983452 0 0) 0x7f96ac003520 con 0x7f96c00a49a0 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f96c00a0f60 con 0x7f96c00a49a0 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b0001550 con 0x7f96c00a49a0 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): dump: 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.311 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 127809534 0 0) 0x7f96c00a1f30 con 0x7f96c013e380 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f96ac003520 con 0x7f96c013e380 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b8001c00 con 0x7f96c013e380 2024-05-08T15:49:34.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): dump: 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.313 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 808521436 0 0) 0x7f96c009f9c0 con 0x7f96c00a5430 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f96c00a1f30 con 0x7f96c00a5430 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b4001770 con 0x7f96c00a5430 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient(hunting): dump: 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.314 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4282745239 0 0) 0x7f96c00a0f60 con 0x7f96c00a49a0 2024-05-08T15:49:34.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.81:6791/0 conn(0x7f96c00a5430 legacy=0x7f96c01d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.7:6789/0 conn(0x7f96c013e380 legacy=0x7f96c01d5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 monclient: found mon.b 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c009e710 con 0x7f96c00a49a0 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96c51db640 5 monclient: authenticate success, global_id 94313 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96c51db640 10 monclient: _renew_subs 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96c51db640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f96c00a0690 con 0x7f96c00a49a0 2024-05-08T15:49:34.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96c51db640 10 monclient: _renew_subs 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96c51db640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f96c01c9c20 con 0x7f96c00a49a0 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f96b00038a0 con 0x7f96c00a49a0 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b0003e10 con 0x7f96c00a49a0 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96bd59a640 4 set_mon_vals no callback set 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.299+0000 7f96a77fe640 10 monclient: dump: 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.317 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f96b0004030 con 0x7f96c00a49a0 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96c51db640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f96a00065c0 con 0x7f96a0002860 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(101..101 src has 1..101) ==== 3898+0+0 (unknown 3858878525 0 0) 0x7f96b0046e70 con 0x7f96c00a49a0 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96be59c640 10 monclient: get_auth_request con 0x7f96ac03a460 auth_method 0 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4098788403 0 0) 0x7f96a00065c0 con 0x7f96a0002860 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 10 monclient(hunting): my global_id is 75156 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c00a0f60 con 0x7f96a0002860 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3648890175 0 0) 0x7f96c00a0f60 con 0x7f96a0002860 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f96a00065c0 con 0x7f96a0002860 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b8004ea0 con 0x7f96a0002860 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:34.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.300+0000 7f96a77fe640 10 monclient: dump: 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.320 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.301+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1049412890 0 0) 0x7f96a00065c0 con 0x7f96a0002860 2024-05-08T15:49:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.301+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96a000a150 con 0x7f96a0002860 2024-05-08T15:49:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.301+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f96b8006340 con 0x7f96a0002860 2024-05-08T15:49:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.301+0000 7f96a77fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.301+0000 7f96a77fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.301+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.7:6792/0 conn(0x7f96a0002860 legacy=0x7f96a0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.301+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.7:6792/0 conn(0x7f96a0002860 legacy=0x7f96a0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.307+0000 7f96c51db640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.307+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f96a0009e40 con 0x7f96a0009100 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3950963306 0 0) 0x7f96a0009e40 con 0x7f96a0009100 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient(hunting): my global_id is 75165 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96a00065c0 con 0x7f96a0009100 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1715707496 0 0) 0x7f96a00065c0 con 0x7f96a0009100 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f96a0009e40 con 0x7f96a0009100 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f96b80054d0 con 0x7f96a0009100 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:34.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient: dump: 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.324 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1979072854 0 0) 0x7f96a0009e40 con 0x7f96a0009100 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f96a000c3d0 con 0x7f96a0009100 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7841 (unknown 0 0 3909095132) 0x7f96b8005750 con 0x7f96a0009100 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.7:6792/0 conn(0x7f96a0009100 legacy=0x7f96a0009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.308+0000 7f96a77fe640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.7:6792/0 conn(0x7f96a0009100 legacy=0x7f96a0009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:34.325 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 24, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 166622, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:34.326 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:34.327 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:34.328 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.329 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.330 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.331 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:34.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.336 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:34.338 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:34.340 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.342 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.344 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.344 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:34.344 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.81:6812/847627118 conn(0x7f96ac03a460 legacy=0x7f96ac03c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 10 monclient: shutdown 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 >> v1:172.21.15.7:6790/0 conn(0x7f96c00a49a0 legacy=0x7f96c01c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96bf59e640 1 -- 172.21.15.7:0/2768363697 reap_dead start 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96bf59e640 1 -- 172.21.15.7:0/2768363697 reap_dead start 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 shutdown_connections 2024-05-08T15:49:34.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 >> 172.21.15.7:0/2768363697 conn(0x7f96c00a9fa0 msgr2=0x7f96c0138a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 shutdown_connections 2024-05-08T15:49:34.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.309+0000 7f96c51db640 1 -- 172.21.15.7:0/2768363697 wait complete. 2024-05-08T15:49:34.346 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:49:34.346 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-05-08T15:49:34.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:49:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.454+0000 7ff8fc8e0640 10 monclient: get_monmap_and_config 2024-05-08T15:49:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.454+0000 7ff8fc8e0640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.454+0000 7ff8fc8e0640 10 monclient: monmap: 2024-05-08T15:49:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.450 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:34.455390+0000 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:34.455390+0000 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.451 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 1 Processor -- start 2024-05-08T15:49:34.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 1 -- start start 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient: init 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient: _add_conns ranks=[6,1,5,3,4,2,7,8,0] 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient(hunting): picked mon.noname-d con 0x7ff8f8137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient(hunting): picked mon.noname-f con 0x7ff8f80a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient(hunting): picked mon.noname-h con 0x7ff8f80a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f80a2c70 con 0x7ff8f80a49a0 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f80a2f50 con 0x7ff8f80a41b0 2024-05-08T15:49:34.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f80a1670 con 0x7ff8f8137940 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8fc8e0640 10 monclient(hunting): authenticate will time out at 982.570517s 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8f659c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7ff8f8137940 0x7ff8f8137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:34912/0 (socket says 172.21.15.7:34912) 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.455+0000 7ff8f659c640 1 -- 172.21.15.7:0/3496111447 learned_addr learned my addr 172.21.15.7:0/3496111447 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677168247 0 0) 0x7ff8f80a1670 con 0x7ff8f8137940 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): my global_id is 75174 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8e0003520 con 0x7ff8f8137940 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1841041585 0 0) 0x7ff8f80a2c70 con 0x7ff8f80a49a0 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): my global_id is 5140 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8f80a1670 con 0x7ff8f80a49a0 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218798050 0 0) 0x7ff8f80a2f50 con 0x7ff8f80a41b0 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient(hunting): my global_id is 5135 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8f80a2c70 con 0x7ff8f80a41b0 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 806315036 0 0) 0x7ff8e0003520 con 0x7ff8f8137940 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 >> v1:172.21.15.81:6791/0 conn(0x7ff8f80a41b0 legacy=0x7ff8f8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 >> v1:172.21.15.81:6789/0 conn(0x7ff8f80a49a0 legacy=0x7ff8f813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 monclient: found mon.noname-d 2024-05-08T15:49:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f80a39f0 con 0x7ff8f8137940 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8e80017b0 con 0x7ff8f8137940 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8fc8e0640 5 monclient: authenticate success, global_id 75174 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8fc8e0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: dump: 2024-05-08T15:49:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.468 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8fc8e0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff8e8001a10 con 0x7ff8f8137940 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 1 -- 172.21.15.7:0/3496111447 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8e8004be0 con 0x7ff8f8137940 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8fc8e0640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8fc8e0640 4 set_mon_vals no callback set 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.456+0000 7ff8f559a640 10 monclient: dump: 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.469 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 10 monclient: shutdown 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3496111447 >> v1:172.21.15.7:6792/0 conn(0x7ff8f8137940 legacy=0x7ff8f8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3496111447 shutdown_connections 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3496111447 >> 172.21.15.7:0/3496111447 conn(0x7ff8f80a9fa0 msgr2=0x7ff8f80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3496111447 shutdown_connections 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3496111447 wait complete. 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.457+0000 7ff8fc8e0640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 10 monclient: monmap: 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.471 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 1 Processor -- start 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 1 -- start start 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 10 monclient: init 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 10 monclient: _add_conns ranks=[3,1,6,4,0,5,8,7,2] 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 10 monclient(hunting): picked mon.noname-g con 0x7ff8f80a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 10 monclient(hunting): picked mon.noname-f con 0x7ff8f80a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 10 monclient(hunting): picked mon.noname-d con 0x7ff8f8137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f80a0f60 con 0x7ff8f80a49a0 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.458+0000 7ff8fc8e0640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f8135cc0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8f8135fa0 con 0x7ff8f8137940 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 10 monclient(hunting): authenticate will time out at 982.573618s 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8f6d9d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7ff8f8137940 0x7ff8f81d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:34924/0 (socket says 172.21.15.7:34924) 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8f6d9d640 1 -- 172.21.15.7:0/3098414873 learned_addr learned my addr 172.21.15.7:0/3098414873 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959066729 0 0) 0x7ff8f8135fa0 con 0x7ff8f8137940 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): my global_id is 75183 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8d8003520 con 0x7ff8f8137940 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 81294473 0 0) 0x7ff8f8135cc0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): my global_id is 75189 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8f8135fa0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1342168325 0 0) 0x7ff8f80a1670 con 0x7ff8f80a49a0 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): my global_id is 5149 2024-05-08T15:49:34.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8f8135cc0 con 0x7ff8f80a49a0 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3930565828 0 0) 0x7ff8f8135fa0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8f80a1670 con 0x7ff8f80a41b0 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2898043215 0 0) 0x7ff8d8003520 con 0x7ff8f8137940 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8f8135fa0 con 0x7ff8f8137940 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8e8001bc0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.475 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8f0001ca0 con 0x7ff8f8137940 2024-05-08T15:49:34.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.477 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1111839786 0 0) 0x7ff8f8135cc0 con 0x7ff8f80a49a0 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8d8003520 con 0x7ff8f80a49a0 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8ec001a50 con 0x7ff8f80a49a0 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient(hunting): dump: 2024-05-08T15:49:34.478 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.479 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1519151548 0 0) 0x7ff8f80a1670 con 0x7ff8f80a41b0 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.7:6792/0 conn(0x7ff8f8137940 legacy=0x7ff8f81d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.81:6789/0 conn(0x7ff8f80a49a0 legacy=0x7ff8f81328c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 monclient: found mon.g 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f80a1f30 con 0x7ff8f80a41b0 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 5 monclient: authenticate success, global_id 75189 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 10 monclient: _renew_subs 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f8133cb0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 10 monclient: _renew_subs 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.459+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff8f8134a30 con 0x7ff8f80a41b0 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff8e8004dd0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8e6ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8e8001a10 con 0x7ff8f80a41b0 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8f4d99640 4 set_mon_vals no callback set 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8e6ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:34.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8e6ffd640 10 monclient: dump: 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.482 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8fc8e0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff8dc0065c0 con 0x7ff8dc002860 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.460+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7ff8e80104a0 con 0x7ff8f80a41b0 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(101..101 src has 1..101) ==== 3898+0+0 (unknown 3858878525 0 0) 0x7ff8e8046d60 con 0x7ff8f80a41b0 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785239788 0 0) 0x7ff8dc0065c0 con 0x7ff8dc002860 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 10 monclient(hunting): my global_id is 94292 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8f80a1670 con 0x7ff8dc002860 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8f6d9d640 10 monclient: get_auth_request con 0x7ff8d803a550 auth_method 0 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2736667958 0 0) 0x7ff8f80a1670 con 0x7ff8dc002860 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8dc0065c0 con 0x7ff8dc002860 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8ec001d40 con 0x7ff8dc002860 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 10 monclient: dump: 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.484 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1210023872 0 0) 0x7ff8dc0065c0 con 0x7ff8dc002860 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.461+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff8dc00a150 con 0x7ff8dc002860 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.462+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff8ec005be0 con 0x7ff8dc002860 2024-05-08T15:49:34.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.462+0000 7ff8e6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.462+0000 7ff8e6ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.462+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.7:6793/0 conn(0x7ff8dc002860 legacy=0x7ff8dc004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.462+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.7:6793/0 conn(0x7ff8dc002860 legacy=0x7ff8dc004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8fc8e0640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff8dc009e20 con 0x7ff8dc0090e0 2024-05-08T15:49:34.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287920190 0 0) 0x7ff8dc009e20 con 0x7ff8dc0090e0 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 10 monclient(hunting): my global_id is 94301 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8dc0065c0 con 0x7ff8dc0090e0 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2501187037 0 0) 0x7ff8dc0065c0 con 0x7ff8dc0090e0 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8dc009e20 con 0x7ff8dc0090e0 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8ec0042a0 con 0x7ff8dc0090e0 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 10 monclient: dump: 2024-05-08T15:49:34.488 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.489 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3987075053 0 0) 0x7ff8dc009e20 con 0x7ff8dc0090e0 2024-05-08T15:49:34.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.469+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff8dc00c390 con 0x7ff8dc0090e0 2024-05-08T15:49:34.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7500 (unknown 0 0 401327085) 0x7ff8ec005760 con 0x7ff8dc0090e0 2024-05-08T15:49:34.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8e6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8e6ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.7:6793/0 conn(0x7ff8dc0090e0 legacy=0x7ff8dc009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8e6ffd640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.7:6793/0 conn(0x7ff8dc0090e0 legacy=0x7ff8dc009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.491 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 24, 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 25021, 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:34.492 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.493 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.494 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:34.495 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.496 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.497 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:34.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.501 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.502 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:34.503 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.504 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.505 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.506 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.507 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.508 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.509 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.510 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.511 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.512 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.513 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.514 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:34.515 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.81:6812/847627118 conn(0x7ff8d803a550 legacy=0x7ff8d803ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 10 monclient: shutdown 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 >> v1:172.21.15.81:6790/0 conn(0x7ff8f80a41b0 legacy=0x7ff8f81321b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8f6d9d640 1 -- 172.21.15.7:0/3098414873 reap_dead start 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8f6d9d640 1 -- 172.21.15.7:0/3098414873 reap_dead start 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 shutdown_connections 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 >> 172.21.15.7:0/3098414873 conn(0x7ff8f80a9fa0 msgr2=0x7ff8f8130c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 shutdown_connections 2024-05-08T15:49:34.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.470+0000 7ff8fc8e0640 1 -- 172.21.15.7:0/3098414873 wait complete. 2024-05-08T15:49:34.518 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:49:34.518 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-05-08T15:49:34.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:49:34.579 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:34.579 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35407]: starting. 2024-05-08T15:49:34.579 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35407]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35308 2024-05-08T15:49:34.579 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35407]: shutting down. 2024-05-08T15:49:34.581 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:34.581 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:34.581 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:49:34.583 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:49:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.633+0000 7fe01bc38640 10 monclient: get_monmap_and_config 2024-05-08T15:49:34.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.633+0000 7fe01bc38640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.633+0000 7fe01bc38640 10 monclient: monmap: 2024-05-08T15:49:34.630 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.630 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.630 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:34.634173+0000 2024-05-08T15:49:34.630 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:34.634173+0000 2024-05-08T15:49:34.630 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.631 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.631 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.631 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.631 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.631 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.632 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.632 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.632 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.632 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.632 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.632 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.633+0000 7fe01bc38640 1 Processor -- start 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 1 -- start start 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient: init 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient: _add_conns ranks=[8,3,1,6,7,2,4,5,0] 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient(hunting): picked mon.noname-e con 0x7fe0140a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient(hunting): picked mon.noname-g con 0x7fe0140a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient(hunting): picked mon.noname-f con 0x7fe01413e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:34.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0140a1660 con 0x7fe01413e310 2024-05-08T15:49:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0140a28f0 con 0x7fe0140a5780 2024-05-08T15:49:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0140a2bd0 con 0x7fe0140a53a0 2024-05-08T15:49:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe01bc38640 10 monclient(hunting): authenticate will time out at 982.749369s 2024-05-08T15:49:34.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe0199d4640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fe0140a53a0 0x7fe0140a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:52764/0 (socket says 172.21.15.7:52764) 2024-05-08T15:49:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.634+0000 7fe0199d4640 1 -- 172.21.15.7:0/3783046534 learned_addr learned my addr 172.21.15.7:0/3783046534 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776523052 0 0) 0x7fe0140a2bd0 con 0x7fe0140a53a0 2024-05-08T15:49:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): my global_id is 94319 2024-05-08T15:49:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe000003520 con 0x7fe0140a53a0 2024-05-08T15:49:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525310391 0 0) 0x7fe0140a1660 con 0x7fe01413e310 2024-05-08T15:49:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): my global_id is 5167 2024-05-08T15:49:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0140a2bd0 con 0x7fe01413e310 2024-05-08T15:49:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296425447 0 0) 0x7fe0140a28f0 con 0x7fe0140a5780 2024-05-08T15:49:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient(hunting): my global_id is 75216 2024-05-08T15:49:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0140a1660 con 0x7fe0140a5780 2024-05-08T15:49:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2506159870 0 0) 0x7fe000003520 con 0x7fe0140a53a0 2024-05-08T15:49:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 >> v1:172.21.15.81:6790/0 conn(0x7fe0140a5780 legacy=0x7fe01413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 >> v1:172.21.15.81:6789/0 conn(0x7fe01413e310 legacy=0x7fe014140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 monclient: found mon.noname-e 2024-05-08T15:49:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:49:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0140a6730 con 0x7fe0140a53a0 2024-05-08T15:49:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 5 monclient: authenticate success, global_id 94319 2024-05-08T15:49:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe0100017b0 con 0x7fe0140a53a0 2024-05-08T15:49:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: dump: 2024-05-08T15:49:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.659 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe010001ad0 con 0x7fe0140a53a0 2024-05-08T15:49:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 1 -- 172.21.15.7:0/3783046534 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe010004be0 con 0x7fe0140a53a0 2024-05-08T15:49:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 4 set_mon_vals no callback set 2024-05-08T15:49:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe0189d2640 10 monclient: dump: 2024-05-08T15:49:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.661 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.662 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.662 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.662 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.662 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.662 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.662 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.662 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.663 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.663 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.663 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.663 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 10 monclient: shutdown 2024-05-08T15:49:34.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 1 -- 172.21.15.7:0/3783046534 >> v1:172.21.15.7:6793/0 conn(0x7fe0140a53a0 legacy=0x7fe0140a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.635+0000 7fe01bc38640 1 -- 172.21.15.7:0/3783046534 shutdown_connections 2024-05-08T15:49:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.636+0000 7fe01bc38640 1 -- 172.21.15.7:0/3783046534 >> 172.21.15.7:0/3783046534 conn(0x7fe0140a9fa0 msgr2=0x7fe0140aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.636+0000 7fe01bc38640 1 -- 172.21.15.7:0/3783046534 shutdown_connections 2024-05-08T15:49:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.636+0000 7fe01bc38640 1 -- 172.21.15.7:0/3783046534 wait complete. 2024-05-08T15:49:34.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.636+0000 7fe01bc38640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.636+0000 7fe01bc38640 10 monclient: monmap: 2024-05-08T15:49:34.665 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.665 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:34.665 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:34.665 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:34.665 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.665 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:34.666 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.666 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.666 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.666 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.667 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.667 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.636+0000 7fe01bc38640 1 Processor -- start 2024-05-08T15:49:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 1 -- start start 2024-05-08T15:49:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient: init 2024-05-08T15:49:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient: _add_conns ranks=[8,2,1,5,6,4,0,7,3] 2024-05-08T15:49:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient(hunting): picked mon.noname-e con 0x7fe0140a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:34.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient(hunting): picked mon.noname-b con 0x7fe0140a5780 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient(hunting): picked mon.noname-f con 0x7fe01413e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:49:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe014134e90 con 0x7fe01413e310 2024-05-08T15:49:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0140a1e00 con 0x7fe0140a5780 2024-05-08T15:49:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe014134860 con 0x7fe0140a53a0 2024-05-08T15:49:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe0199d4640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fe0140a53a0 0x7fe01413bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:52776/0 (socket says 172.21.15.7:52776) 2024-05-08T15:49:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe01bc38640 10 monclient(hunting): authenticate will time out at 982.752216s 2024-05-08T15:49:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe0199d4640 1 -- 172.21.15.7:0/1435038315 learned_addr learned my addr 172.21.15.7:0/1435038315 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1026788685 0 0) 0x7fe014134860 con 0x7fe0140a53a0 2024-05-08T15:49:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): my global_id is 94328 2024-05-08T15:49:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdff8003520 con 0x7fe0140a53a0 2024-05-08T15:49:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2932311264 0 0) 0x7fe0140a1660 con 0x7fe0140a5780 2024-05-08T15:49:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 10 monclient(hunting): my global_id is 94331 2024-05-08T15:49:34.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe014134860 con 0x7fe0140a5780 2024-05-08T15:49:34.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.637+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157276531 0 0) 0x7fe014134e90 con 0x7fe01413e310 2024-05-08T15:49:34.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): my global_id is 5185 2024-05-08T15:49:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0140a1660 con 0x7fe01413e310 2024-05-08T15:49:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2160478973 0 0) 0x7fdff8003520 con 0x7fe0140a53a0 2024-05-08T15:49:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe014134e90 con 0x7fe0140a53a0 2024-05-08T15:49:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe010004b20 con 0x7fe0140a53a0 2024-05-08T15:49:34.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): dump: 2024-05-08T15:49:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.674 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.675 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.676 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.677 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2517455467 0 0) 0x7fe014134860 con 0x7fe0140a5780 2024-05-08T15:49:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdff8003520 con 0x7fe0140a5780 2024-05-08T15:49:34.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe004001990 con 0x7fe0140a5780 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): dump: 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.678 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.679 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2528120299 0 0) 0x7fe0140a1660 con 0x7fe01413e310 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe014134860 con 0x7fe01413e310 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe00c001bc0 con 0x7fe01413e310 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient(hunting): dump: 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.680 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1314627043 0 0) 0x7fe014134e90 con 0x7fe0140a53a0 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.7:6790/0 conn(0x7fe0140a5780 legacy=0x7fe01413c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.81:6789/0 conn(0x7fe01413e310 legacy=0x7fe0141d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 monclient: found mon.e 2024-05-08T15:49:34.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01413cb70 con 0x7fe0140a53a0 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe01bc38640 5 monclient: authenticate success, global_id 94328 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe01bc38640 10 monclient: _renew_subs 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe01bc38640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe014134580 con 0x7fe0140a53a0 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe01bc38640 10 monclient: _renew_subs 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe01bc38640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fe014133210 con 0x7fe0140a53a0 2024-05-08T15:49:34.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe010001ad0 con 0x7fe0140a53a0 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe010001c80 con 0x7fe0140a53a0 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00bfff640 4 set_mon_vals no callback set 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.638+0000 7fe00a7fc640 10 monclient: dump: 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.683 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe01bc38640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdffc0065c0 con 0x7fdffc002860 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fe0100106c0 con 0x7fe0140a53a0 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(102..102 src has 1..102) ==== 3506+0+0 (unknown 2760887463 0 0) 0x7fe010046cf0 con 0x7fe0140a53a0 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 700055131 0 0) 0x7fdffc0065c0 con 0x7fdffc002860 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 10 monclient(hunting): my global_id is 5194 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe014134e90 con 0x7fdffc002860 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.639+0000 7fe01a1d5640 10 monclient: get_auth_request con 0x7fdff803a3d0 auth_method 0 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.640+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4025376248 0 0) 0x7fe014134e90 con 0x7fdffc002860 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.640+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdffc0065c0 con 0x7fdffc002860 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.640+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe004001d00 con 0x7fdffc002860 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.640+0000 7fe00a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.640+0000 7fe00a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.640+0000 7fe00a7fc640 10 monclient: dump: 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.685 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.686 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.659+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 657472894 0 0) 0x7fdffc0065c0 con 0x7fdffc002860 2024-05-08T15:49:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.659+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdffc00a150 con 0x7fdffc002860 2024-05-08T15:49:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.659+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe004006090 con 0x7fdffc002860 2024-05-08T15:49:34.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.659+0000 7fe00a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.659+0000 7fe00a7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.659+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.81:6789/0 conn(0x7fdffc002860 legacy=0x7fdffc004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.659+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.81:6789/0 conn(0x7fdffc002860 legacy=0x7fdffc004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.666+0000 7fe01bc38640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.666+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fdffc009e40 con 0x7fdffc009100 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143488473 0 0) 0x7fdffc009e40 con 0x7fdffc009100 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient(hunting): my global_id is 5203 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdffc0065c0 con 0x7fdffc009100 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2390017599 0 0) 0x7fdffc0065c0 con 0x7fdffc009100 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdffc009e40 con 0x7fdffc009100 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe0040046b0 con 0x7fdffc009100 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient: dump: 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.690 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2662826725 0 0) 0x7fdffc009e40 con 0x7fdffc009100 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdffc00c3d0 con 0x7fdffc009100 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7673 (unknown 0 0 607073680) 0x7fe004009080 con 0x7fdffc009100 2024-05-08T15:49:34.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.81:6789/0 conn(0x7fdffc009100 legacy=0x7fdffc009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.667+0000 7fe00a7fc640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.81:6789/0 conn(0x7fdffc009100 legacy=0x7fdffc009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.692 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:34.692 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 24, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 166982, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:34.693 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.694 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.695 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:34.696 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.697 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.699 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.700 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.703 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.704 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:34.707 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.708 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.710 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.711 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.712 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:34.713 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:34.714 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.81:6812/847627118 conn(0x7fdff803a3d0 legacy=0x7fdff803c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 10 monclient: shutdown 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 >> v1:172.21.15.7:6793/0 conn(0x7fe0140a53a0 legacy=0x7fe01413bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01a1d5640 1 -- 172.21.15.7:0/1435038315 reap_dead start 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01a1d5640 1 -- 172.21.15.7:0/1435038315 reap_dead start 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 shutdown_connections 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 >> 172.21.15.7:0/1435038315 conn(0x7fe0140a9fa0 msgr2=0x7fe0140a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 shutdown_connections 2024-05-08T15:49:34.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.668+0000 7fe01bc38640 1 -- 172.21.15.7:0/1435038315 wait complete. 2024-05-08T15:49:34.717 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:49:34.717 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-05-08T15:49:34.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:49:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.824+0000 7f27c20ba640 10 monclient: get_monmap_and_config 2024-05-08T15:49:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.824+0000 7f27c20ba640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.824+0000 7f27c20ba640 10 monclient: monmap: 2024-05-08T15:49:34.821 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:34.825913+0000 2024-05-08T15:49:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:34.825913+0000 2024-05-08T15:49:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.822 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.823 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.824 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.824 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.825+0000 7f27c20ba640 1 Processor -- start 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.825+0000 7f27c20ba640 1 -- start start 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.825+0000 7f27c20ba640 10 monclient: init 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 10 monclient: _add_conns ranks=[3,6,8,0,4,7,2,1,5] 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 10 monclient(hunting): picked mon.noname-g con 0x7f27bc137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 10 monclient(hunting): picked mon.noname-d con 0x7f27bc0a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 10 monclient(hunting): picked mon.noname-e con 0x7f27bc0a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc0a2c70 con 0x7f27bc137940 2024-05-08T15:49:34.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc0a2f50 con 0x7f27bc0a49a0 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc0a1670 con 0x7f27bc0a41b0 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 10 monclient(hunting): authenticate will time out at 982.940867s 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27baffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f27bc0a49a0 0x7f27bc13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:53892/0 (socket says 172.21.15.7:53892) 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27baffd640 1 -- 172.21.15.7:0/2386579761 learned_addr learned my addr 172.21.15.7:0/2386579761 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27bbfff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f27bc0a41b0 0x7f27bc140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:46322/0 (socket says 172.21.15.7:46322) 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3204156521 0 0) 0x7f27bc0a2f50 con 0x7f27bc0a49a0 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): my global_id is 75201 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27a4003520 con 0x7f27bc0a49a0 2024-05-08T15:49:34.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581812674 0 0) 0x7f27bc0a1670 con 0x7f27bc0a41b0 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): my global_id is 94337 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27bc0a2f50 con 0x7f27bc0a41b0 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632830663 0 0) 0x7f27bc0a2c70 con 0x7f27bc137940 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient(hunting): my global_id is 75234 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27bc0a1670 con 0x7f27bc137940 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 868619873 0 0) 0x7f27bc0a2f50 con 0x7f27bc0a41b0 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 >> v1:172.21.15.7:6792/0 conn(0x7f27bc0a49a0 legacy=0x7f27bc13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 >> v1:172.21.15.81:6790/0 conn(0x7f27bc137940 legacy=0x7f27bc137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 monclient: found mon.noname-e 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc0a39f0 con 0x7f27bc0a41b0 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 5 monclient: authenticate success, global_id 94337 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f27b4001a90 con 0x7f27bc0a41b0 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27ba7fc640 10 monclient: dump: 2024-05-08T15:49:34.845 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.846 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.826+0000 7f27c20ba640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f27b4001df0 con 0x7f27bc0a41b0 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27ba7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27ba7fc640 1 -- 172.21.15.7:0/2386579761 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f27b4004ea0 con 0x7f27bc0a41b0 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27ba7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 4 set_mon_vals no callback set 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27ba7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27ba7fc640 10 monclient: dump: 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.847 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 10 monclient: shutdown 2024-05-08T15:49:34.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 1 -- 172.21.15.7:0/2386579761 >> v1:172.21.15.7:6793/0 conn(0x7f27bc0a41b0 legacy=0x7f27bc140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 1 -- 172.21.15.7:0/2386579761 shutdown_connections 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 1 -- 172.21.15.7:0/2386579761 >> 172.21.15.7:0/2386579761 conn(0x7f27bc0a9fa0 msgr2=0x7f27bc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 1 -- 172.21.15.7:0/2386579761 shutdown_connections 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.827+0000 7f27c20ba640 1 -- 172.21.15.7:0/2386579761 wait complete. 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.828+0000 7f27c20ba640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.828+0000 7f27c20ba640 10 monclient: monmap: 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.849 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.828+0000 7f27c20ba640 1 Processor -- start 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.828+0000 7f27c20ba640 1 -- start start 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.828+0000 7f27c20ba640 10 monclient: init 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:34.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient: _add_conns ranks=[3,4,6,7,2,5,1,0,8] 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient(hunting): picked mon.noname-g con 0x7f27bc0a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient(hunting): picked mon.noname-c con 0x7f27bc0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient(hunting): picked mon.noname-d con 0x7f27bc137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc0a1f30 con 0x7f27bc0a41b0 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc134cc0 con 0x7f27bc0a49a0 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27bc0a0f60 con 0x7f27bc137940 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient(hunting): authenticate will time out at 982.943712s 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27baffd640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7f27bc0a49a0 legacy=0x7f27bc1d1680 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27bbfff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f27bc137940 0x7f27bc1d4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:53902/0 (socket says 172.21.15.7:53902) 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27bbfff640 1 -- 172.21.15.7:0/1976695493 learned_addr learned my addr 172.21.15.7:0/1976695493 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1176783864 0 0) 0x7f27bc0a0f60 con 0x7f27bc137940 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): my global_id is 75210 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27a8003520 con 0x7f27bc137940 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510227295 0 0) 0x7f27bc0a1f30 con 0x7f27bc0a41b0 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): my global_id is 75243 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27bc0a0f60 con 0x7f27bc0a41b0 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2409434381 0 0) 0x7f27a8003520 con 0x7f27bc137940 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f27bc0a1f30 con 0x7f27bc137940 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f27b4001a50 con 0x7f27bc137940 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): dump: 2024-05-08T15:49:34.852 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1525567637 0 0) 0x7f27bc0a0f60 con 0x7f27bc0a41b0 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f27a8003520 con 0x7f27bc0a41b0 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f27ac001970 con 0x7f27bc0a41b0 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient(hunting): dump: 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.854 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3465304220 0 0) 0x7f27bc0a1f30 con 0x7f27bc137940 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.7:6791/0 conn(0x7f27bc0a49a0 legacy=0x7f27bc1d1680 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.81:6790/0 conn(0x7f27bc0a41b0 legacy=0x7f27bc136fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: _finish_hunting 0 2024-05-08T15:49:34.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 monclient: found mon.d 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc1337a0 con 0x7f27bc137940 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: _finish_auth 0 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: _check_auth_tickets 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 5 monclient: authenticate success, global_id 75210 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient: _renew_subs 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc1357b0 con 0x7f27bc137940 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient: _renew_subs 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f27bc1376d0 con 0x7f27bc137940 2024-05-08T15:49:34.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f27b40051b0 con 0x7f27bc137940 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f27b40056a0 con 0x7f27bc137940 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27a3fff640 10 monclient: dump: 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.857 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.829+0000 7f27b9ffb640 4 set_mon_vals no callback set 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.830+0000 7f27c20ba640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.830+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f27bc0753d0 con 0x7f27bc134540 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.830+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f27b4010aa0 con 0x7f27bc137940 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.830+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(102..102 src has 1..102) ==== 3506+0+0 (unknown 2760887463 0 0) 0x7f27b4047110 con 0x7f27bc137940 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4006397971 0 0) 0x7f27bc0753d0 con 0x7f27bc134540 2024-05-08T15:49:34.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient(hunting): my global_id is 75252 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27bc0a1f30 con 0x7f27bc134540 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27baffd640 10 monclient: get_auth_request con 0x7f27a803a080 auth_method 0 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1076630794 0 0) 0x7f27bc0a1f30 con 0x7f27bc134540 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f27bc0753d0 con 0x7f27bc134540 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f27ac003930 con 0x7f27bc134540 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient: dump: 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.859 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2177276798 0 0) 0x7f27bc0753d0 con 0x7f27bc134540 2024-05-08T15:49:34.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27bc0008d0 con 0x7f27bc134540 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f27ac004640 con 0x7f27bc134540 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.81:6790/0 conn(0x7f27bc134540 legacy=0x7f27bc09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.831+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.81:6790/0 conn(0x7f27bc134540 legacy=0x7f27bc09ba50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27c20ba640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f27bc09c870 con 0x7f27bc1cc360 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4066609811 0 0) 0x7f27bc09c870 con 0x7f27bc1cc360 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27a3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27a3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27a3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27a3fff640 10 monclient(hunting): my global_id is 75261 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.838+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f27bc0753d0 con 0x7f27bc1cc360 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3688278138 0 0) 0x7f27bc0753d0 con 0x7f27bc1cc360 2024-05-08T15:49:34.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f27bc09c870 con 0x7f27bc1cc360 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f27ac004210 con 0x7f27bc1cc360 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 10 monclient: dump: 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:34.862 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3158564127 0 0) 0x7f27bc09c870 con 0x7f27bc1cc360 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f27bc133070 con 0x7f27bc1cc360 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7671 (unknown 0 0 483151988) 0x7f27ac0047f0 con 0x7f27bc1cc360 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.81:6790/0 conn(0x7f27bc1cc360 legacy=0x7f27bc09c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27a3fff640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.81:6790/0 conn(0x7f27bc1cc360 legacy=0x7f27bc09c160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 24, 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 167128, 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:34.864 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.865 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.866 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:34.867 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:34.868 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.869 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.870 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.871 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.872 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.873 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:34.875 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:34.877 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:34.879 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.880 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:34.881 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:49:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:34.882 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:34.882 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.81:6812/847627118 conn(0x7f27a803a080 legacy=0x7f27a803c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27c20ba640 10 monclient: shutdown 2024-05-08T15:49:34.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27c20ba640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 >> v1:172.21.15.7:6792/0 conn(0x7f27bc137940 legacy=0x7f27bc1d4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27bbfff640 1 -- 172.21.15.7:0/1976695493 reap_dead start 2024-05-08T15:49:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27bbfff640 1 -- 172.21.15.7:0/1976695493 reap_dead start 2024-05-08T15:49:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 shutdown_connections 2024-05-08T15:49:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.839+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 >> 172.21.15.7:0/1976695493 conn(0x7f27bc0a9fa0 msgr2=0x7f27bc130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.840+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 shutdown_connections 2024-05-08T15:49:34.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.840+0000 7f27c20ba640 1 -- 172.21.15.7:0/1976695493 wait complete. 2024-05-08T15:49:34.884 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:49:34.884 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-05-08T15:49:34.885 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:49:34.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.993+0000 7f081082c640 10 monclient: get_monmap_and_config 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.993+0000 7f081082c640 10 monclient: build_initial_monmap 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.993+0000 7f081082c640 10 monclient: monmap: 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:49:34.994221+0000 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:49:34.994221+0000 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:34.990 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.993+0000 7f081082c640 1 Processor -- start 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.993+0000 7f081082c640 1 -- start start 2024-05-08T15:49:34.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.993+0000 7f081082c640 10 monclient: init 2024-05-08T15:49:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 10 monclient: _add_conns ranks=[6,5,4,7,1,2,0,8,3] 2024-05-08T15:49:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 10 monclient(hunting): picked mon.noname-d con 0x7f080c0a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:49:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 10 monclient(hunting): picked mon.noname-h con 0x7f080c0a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:49:35.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 10 monclient(hunting): picked mon.noname-c con 0x7f080c13e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c0a1660 con 0x7f080c13e310 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c0a28f0 con 0x7f080c0a5780 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c0a2bd0 con 0x7f080c0a53a0 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f081082c640 10 monclient(hunting): authenticate will time out at 983.109162s 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080ad9d640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7f080c13e310 legacy=0x7f080c140700 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080a59c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f080c0a53a0 0x7f080c0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:53918/0 (socket says 172.21.15.7:53918) 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080a59c640 1 -- 172.21.15.7:0/3968764067 learned_addr learned my addr 172.21.15.7:0/3968764067 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2962244694 0 0) 0x7f080c0a2bd0 con 0x7f080c0a53a0 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080959a640 10 monclient(hunting): _negotiate 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:35.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080959a640 10 monclient(hunting): my global_id is 75219 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.994+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f4003520 con 0x7f080c0a53a0 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4078525221 0 0) 0x7f07f4003520 con 0x7f080c0a53a0 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 >> v1:172.21.15.81:6791/0 conn(0x7f080c0a5780 legacy=0x7f080c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 >> v1:172.21.15.7:6791/0 conn(0x7f080c13e310 legacy=0x7f080c140700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: _finish_hunting 0 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 monclient: found mon.noname-d 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c0a6730 con 0x7f080c0a53a0 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: _finish_auth 0 2024-05-08T15:49:35.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: _check_auth_tickets 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f07fc0017b0 con 0x7f080c0a53a0 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: dump: 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:35.003 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 5 monclient: authenticate success, global_id 75219 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f07fc001b10 con 0x7f080c0a53a0 2024-05-08T15:49:35.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 1 -- 172.21.15.7:0/3968764067 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f07fc001d40 con 0x7f080c0a53a0 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f080959a640 10 monclient: dump: 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.005 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:35.006 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 10 monclient: get_monmap_and_config success 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 4 set_mon_vals no callback set 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 10 monclient: shutdown 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 1 -- 172.21.15.7:0/3968764067 >> v1:172.21.15.7:6792/0 conn(0x7f080c0a53a0 legacy=0x7f080c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 1 -- 172.21.15.7:0/3968764067 shutdown_connections 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 1 -- 172.21.15.7:0/3968764067 >> 172.21.15.7:0/3968764067 conn(0x7f080c0a9fa0 msgr2=0x7f080c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 1 -- 172.21.15.7:0/3968764067 shutdown_connections 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.995+0000 7f081082c640 1 -- 172.21.15.7:0/3968764067 wait complete. 2024-05-08T15:49:35.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.996+0000 7f081082c640 10 monclient: build_initial_monmap 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.996+0000 7f081082c640 10 monclient: monmap: 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:49:35.008 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.996+0000 7f081082c640 1 Processor -- start 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 1 -- start start 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient: init 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient: _reopen_session rank -1 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient: _add_conns ranks=[7,0,2,6,1,5,8,4,3] 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient(hunting): picked mon.noname-i con 0x7f080c0a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient(hunting): picked mon.noname-a con 0x7f080c0a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient(hunting): picked mon.noname-b con 0x7f080c13e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c1373e0 con 0x7f080c0a5780 2024-05-08T15:49:35.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c0a43c0 con 0x7f080c13e310 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c135990 con 0x7f080c0a53a0 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient(hunting): _renew_subs 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f080a59c640 1 -- >> v1:172.21.15.81:6792/0 conn(0x7f080c0a53a0 legacy=0x7f080c131a50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.81:6792/0 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f081082c640 10 monclient(hunting): authenticate will time out at 983.112170s 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f0809d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f080c0a5780 0x7f080c1d1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:56444/0 (socket says 172.21.15.7:56444) 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f080ad9d640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f080c13e310 0x7f080c1d4f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:39294/0 (socket says 172.21.15.7:39294) 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f0809d9b640 1 -- 172.21.15.7:0/536756877 learned_addr learned my addr 172.21.15.7:0/536756877 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600696129 0 0) 0x7f080c0a28f0 con 0x7f080c0a5780 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): my global_id is 94347 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07ec003520 con 0x7f080c0a5780 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1304101470 0 0) 0x7f080c0a43c0 con 0x7f080c13e310 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 10 monclient(hunting): my global_id is 94340 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.997+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f080c0a28f0 con 0x7f080c13e310 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1231184291 0 0) 0x7f080c0a28f0 con 0x7f080c13e310 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f080c0a43c0 con 0x7f080c13e310 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08040044d0 con 0x7f080c13e310 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient(hunting): dump: 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:35.012 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 251867784 0 0) 0x7f07ec003520 con 0x7f080c0a5780 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f080c0a28f0 con 0x7f080c0a5780 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0800001ad0 con 0x7f080c0a5780 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient(hunting): dump: 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:35.013 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2496273861 0 0) 0x7f080c0a43c0 con 0x7f080c13e310 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.81:6792/0 conn(0x7f080c0a53a0 legacy=0x7f080c131a50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:49:35.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.7:6789/0 conn(0x7f080c0a5780 legacy=0x7f080c1d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: _finish_hunting 0 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 monclient: found mon.b 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c1376c0 con 0x7f080c13e310 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: _finish_auth 0 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: _check_auth_tickets 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f081082c640 5 monclient: authenticate success, global_id 94340 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f081082c640 10 monclient: _renew_subs 2024-05-08T15:49:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f081082c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f080c136830 con 0x7f080c13e310 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f081082c640 10 monclient: _renew_subs 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f081082c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f080c138440 con 0x7f080c13e310 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0804004670 con 0x7f080c13e310 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0804004be0 con 0x7f080c13e310 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f0808d99640 4 set_mon_vals no callback set 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:49:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.998+0000 7f07faffd640 10 monclient: dump: 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f081082c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f07f00065c0 con 0x7f07f0002860 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f0804004e00 con 0x7f080c13e310 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(102..102 src has 1..102) ==== 3506+0+0 (unknown 2760887463 0 0) 0x7f0804046d90 con 0x7f080c13e310 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f0809d9b640 10 monclient: get_auth_request con 0x7f07ec03a040 auth_method 0 2024-05-08T15:49:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3519612600 0 0) 0x7f07f00065c0 con 0x7f07f0002860 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 10 monclient(hunting): my global_id is 5162 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:34.999+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f080c0a43c0 con 0x7f07f0002860 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1155692686 0 0) 0x7f080c0a43c0 con 0x7f07f0002860 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f07f00065c0 con 0x7f07f0002860 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f07fc001aa0 con 0x7f07f0002860 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 10 monclient: dump: 2024-05-08T15:49:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:35.020 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 699625234 0 0) 0x7f07f00065c0 con 0x7f07f0002860 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07f000a150 con 0x7f07f0002860 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f07fc001430 con 0x7f07f0002860 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.81:6791/0 conn(0x7f07f0002860 legacy=0x7f07f0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.000+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.81:6791/0 conn(0x7f07f0002860 legacy=0x7f07f0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f081082c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:49:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f07f0009e20 con 0x7f07f00090e0 2024-05-08T15:49:35.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2367370794 0 0) 0x7f07f0009e20 con 0x7f07f00090e0 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 10 monclient(hunting): _negotiate 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 10 monclient(hunting): my global_id is 5171 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07f00065c0 con 0x7f07f00090e0 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4080047837 0 0) 0x7f07f00065c0 con 0x7f07f00090e0 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f07f0009e20 con 0x7f07f00090e0 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f07fc005340 con 0x7f07f00090e0 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:49:35.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.007+0000 7f07faffd640 10 monclient: dump: 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:49:35.025 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 296210697 0 0) 0x7f07f0009e20 con 0x7f07f00090e0 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f07f000c390 con 0x7f07f00090e0 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7501 (unknown 0 0 1017716937) 0x7f07fc005470 con 0x7f07f00090e0 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f07faffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f07faffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.81:6791/0 conn(0x7f07f00090e0 legacy=0x7f07f0009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:35.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f07faffd640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.81:6791/0 conn(0x7f07f00090e0 legacy=0x7f07f0009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 24, 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 167313, 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:49:35.027 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:35.028 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:35.029 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.030 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:49:35.031 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:49:35.032 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:49:35.033 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.034 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:49:35.039 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:49:35.040 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.041 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.042 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:49:35.043 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:49:35.044 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.045 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:49:35.046 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.047 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:49:35.048 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:49:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:49:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:49:35.049 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:49:35.049 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:49:35.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.81:6812/847627118 conn(0x7f07ec03a040 legacy=0x7f07ec03c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:35.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 10 monclient: shutdown 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 >> v1:172.21.15.7:6790/0 conn(0x7f080c13e310 legacy=0x7f080c1d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f080ad9d640 1 -- 172.21.15.7:0/536756877 reap_dead start 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f080ad9d640 1 -- 172.21.15.7:0/536756877 reap_dead start 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 shutdown_connections 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 >> 172.21.15.7:0/536756877 conn(0x7f080c0a9fa0 msgr2=0x7f080c0a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 shutdown_connections 2024-05-08T15:49:35.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:49:35.008+0000 7f081082c640 1 -- 172.21.15.7:0/536756877 wait complete. 2024-05-08T15:49:35.051 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:49:35.052 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-05-08T15:49:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35477]: starting. 2024-05-08T15:49:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35477]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35476 2024-05-08T15:49:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35477]: created object 0... 2024-05-08T15:49:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35477]: created object 25... 2024-05-08T15:49:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35477]: created object 49... 2024-05-08T15:49:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35477]: finishing. 2024-05-08T15:49:36.675 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35477]: shutting down. 2024-05-08T15:49:37.606 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35478]: starting. 2024-05-08T15:49:37.606 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35478]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35476 2024-05-08T15:49:37.606 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35478]: shutting down. 2024-05-08T15:49:39.705 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:39.705 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35513]: starting. 2024-05-08T15:49:39.705 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35513]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35476 2024-05-08T15:49:39.705 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35513]: created object 0... 2024-05-08T15:49:39.705 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35513]: created object 25... 2024-05-08T15:49:39.706 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35513]: created object 49... 2024-05-08T15:49:39.706 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35513]: finishing. 2024-05-08T15:49:39.706 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35513]: shutting down. 2024-05-08T15:49:39.717 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:39.717 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35515]: starting. 2024-05-08T15:49:39.717 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35515]: listing objects. 2024-05-08T15:49:39.717 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35515]: listed object 0... 2024-05-08T15:49:39.718 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35515]: listed object 25... 2024-05-08T15:49:39.718 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35515]: saw 50 objects 2024-05-08T15:49:39.718 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35515]: shutting down. 2024-05-08T15:49:40.620 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:40.620 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35514]: starting. 2024-05-08T15:49:40.620 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35514]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35476 2024-05-08T15:49:40.620 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35514]: shutting down. 2024-05-08T15:49:40.621 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:40.621 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:40.621 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:49:40.624 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:49:42.733 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35584]: starting. 2024-05-08T15:49:42.733 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35584]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35583 2024-05-08T15:49:42.733 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35584]: created object 0... 2024-05-08T15:49:42.733 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35584]: created object 25... 2024-05-08T15:49:42.733 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35584]: created object 49... 2024-05-08T15:49:42.733 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35584]: finishing. 2024-05-08T15:49:42.734 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35584]: shutting down. 2024-05-08T15:49:43.725 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35585]: starting. 2024-05-08T15:49:43.725 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35585]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35583 2024-05-08T15:49:43.725 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35585]: shutting down. 2024-05-08T15:49:45.814 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:45.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35621]: starting. 2024-05-08T15:49:45.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35621]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35583 2024-05-08T15:49:45.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35621]: created object 0... 2024-05-08T15:49:45.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35621]: created object 25... 2024-05-08T15:49:45.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35621]: created object 49... 2024-05-08T15:49:45.815 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35621]: finishing. 2024-05-08T15:49:45.815 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35621]: shutting down. 2024-05-08T15:49:45.824 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:45.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35623]: starting. 2024-05-08T15:49:45.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35623]: listing objects. 2024-05-08T15:49:45.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35623]: listed object 0... 2024-05-08T15:49:45.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35623]: listed object 25... 2024-05-08T15:49:45.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35623]: saw 50 objects 2024-05-08T15:49:45.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35623]: shutting down. 2024-05-08T15:49:46.739 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:46.739 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35622]: starting. 2024-05-08T15:49:46.739 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35622]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35583 2024-05-08T15:49:46.739 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35622]: shutting down. 2024-05-08T15:49:46.741 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:46.741 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:46.741 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:49:46.744 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:49:48.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35692]: starting. 2024-05-08T15:49:48.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35692]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35691 2024-05-08T15:49:48.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35692]: created object 0... 2024-05-08T15:49:48.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35692]: created object 25... 2024-05-08T15:49:48.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35692]: created object 49... 2024-05-08T15:49:48.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35692]: finishing. 2024-05-08T15:49:48.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35692]: shutting down. 2024-05-08T15:49:49.761 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35693]: starting. 2024-05-08T15:49:49.761 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35693]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35691 2024-05-08T15:49:49.761 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35693]: shutting down. 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35728]: starting. 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35728]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35691 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35728]: created object 0... 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35728]: created object 25... 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35728]: created object 49... 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35728]: finishing. 2024-05-08T15:49:51.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35728]: shutting down. 2024-05-08T15:49:51.881 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:51.881 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35730]: starting. 2024-05-08T15:49:51.881 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35730]: listing objects. 2024-05-08T15:49:51.881 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35730]: listed object 0... 2024-05-08T15:49:51.881 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35730]: listed object 25... 2024-05-08T15:49:51.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35730]: saw 50 objects 2024-05-08T15:49:51.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35730]: shutting down. 2024-05-08T15:49:52.721 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:52.721 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35729]: starting. 2024-05-08T15:49:52.721 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35729]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35691 2024-05-08T15:49:52.721 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35729]: shutting down. 2024-05-08T15:49:52.723 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:52.724 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:52.724 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:49:52.727 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:49:54.824 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35799]: starting. 2024-05-08T15:49:54.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35799]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35798 2024-05-08T15:49:54.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35799]: created object 0... 2024-05-08T15:49:54.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35799]: created object 25... 2024-05-08T15:49:54.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35799]: created object 49... 2024-05-08T15:49:54.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35799]: finishing. 2024-05-08T15:49:54.825 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35799]: shutting down. 2024-05-08T15:49:55.741 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35800]: starting. 2024-05-08T15:49:55.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35800]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35798 2024-05-08T15:49:55.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35800]: shutting down. 2024-05-08T15:49:57.831 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:57.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35835]: starting. 2024-05-08T15:49:57.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35835]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35798 2024-05-08T15:49:57.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35835]: created object 0... 2024-05-08T15:49:57.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35835]: created object 25... 2024-05-08T15:49:57.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35835]: created object 49... 2024-05-08T15:49:57.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35835]: finishing. 2024-05-08T15:49:57.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35835]: shutting down. 2024-05-08T15:49:57.843 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:57.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35837]: starting. 2024-05-08T15:49:57.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35837]: listing objects. 2024-05-08T15:49:57.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35837]: listed object 0... 2024-05-08T15:49:57.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35837]: listed object 25... 2024-05-08T15:49:57.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35837]: saw 50 objects 2024-05-08T15:49:57.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35837]: shutting down. 2024-05-08T15:49:58.763 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:58.763 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35836]: starting. 2024-05-08T15:49:58.764 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35836]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35798 2024-05-08T15:49:58.764 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35836]: shutting down. 2024-05-08T15:49:58.765 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:58.765 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:49:58.765 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:49:58.767 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:00.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35906]: starting. 2024-05-08T15:50:00.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35906]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35905 2024-05-08T15:50:00.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35906]: created object 0... 2024-05-08T15:50:00.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35906]: created object 25... 2024-05-08T15:50:00.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35906]: created object 49... 2024-05-08T15:50:00.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35906]: finishing. 2024-05-08T15:50:00.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[35906]: shutting down. 2024-05-08T15:50:01.772 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35907]: starting. 2024-05-08T15:50:01.772 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35907]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35905 2024-05-08T15:50:01.772 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[35907]: shutting down. 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35942]: starting. 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35942]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-35905 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35942]: created object 0... 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35942]: created object 25... 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35942]: created object 49... 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35942]: finishing. 2024-05-08T15:50:03.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[35942]: shutting down. 2024-05-08T15:50:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:03.821 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35944]: starting. 2024-05-08T15:50:03.821 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35944]: listing objects. 2024-05-08T15:50:03.821 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35944]: listed object 0... 2024-05-08T15:50:03.821 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35944]: listed object 25... 2024-05-08T15:50:03.821 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35944]: saw 50 objects 2024-05-08T15:50:03.821 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[35944]: shutting down. 2024-05-08T15:50:04.724 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:04.724 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35943]: starting. 2024-05-08T15:50:04.724 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35943]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-35905 2024-05-08T15:50:04.724 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[35943]: shutting down. 2024-05-08T15:50:04.725 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:04.726 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:04.726 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:04.729 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:06.837 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36014]: starting. 2024-05-08T15:50:06.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36014]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36013 2024-05-08T15:50:06.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36014]: created object 0... 2024-05-08T15:50:06.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36014]: created object 25... 2024-05-08T15:50:06.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36014]: created object 49... 2024-05-08T15:50:06.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36014]: finishing. 2024-05-08T15:50:06.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36014]: shutting down. 2024-05-08T15:50:07.751 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36015]: starting. 2024-05-08T15:50:07.751 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36015]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36013 2024-05-08T15:50:07.752 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36015]: shutting down. 2024-05-08T15:50:09.850 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:09.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36050]: starting. 2024-05-08T15:50:09.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36050]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36013 2024-05-08T15:50:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36050]: created object 0... 2024-05-08T15:50:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36050]: created object 25... 2024-05-08T15:50:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36050]: created object 49... 2024-05-08T15:50:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36050]: finishing. 2024-05-08T15:50:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36050]: shutting down. 2024-05-08T15:50:09.861 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:09.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36052]: starting. 2024-05-08T15:50:09.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36052]: listing objects. 2024-05-08T15:50:09.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36052]: listed object 0... 2024-05-08T15:50:09.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36052]: listed object 25... 2024-05-08T15:50:09.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36052]: saw 50 objects 2024-05-08T15:50:09.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36052]: shutting down. 2024-05-08T15:50:10.774 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:10.774 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36051]: starting. 2024-05-08T15:50:10.774 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36051]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36013 2024-05-08T15:50:10.774 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36051]: shutting down. 2024-05-08T15:50:10.776 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:10.776 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:10.776 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:10.780 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36121]: starting. 2024-05-08T15:50:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36121]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36120 2024-05-08T15:50:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36121]: created object 0... 2024-05-08T15:50:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36121]: created object 25... 2024-05-08T15:50:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36121]: created object 49... 2024-05-08T15:50:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36121]: finishing. 2024-05-08T15:50:12.814 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36121]: shutting down. 2024-05-08T15:50:13.736 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36122]: starting. 2024-05-08T15:50:13.736 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36122]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36120 2024-05-08T15:50:13.736 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36122]: shutting down. 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36157]: starting. 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36157]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36120 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36157]: created object 0... 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36157]: created object 25... 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36157]: created object 49... 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36157]: finishing. 2024-05-08T15:50:15.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36157]: shutting down. 2024-05-08T15:50:15.843 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:15.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36159]: starting. 2024-05-08T15:50:15.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36159]: listing objects. 2024-05-08T15:50:15.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36159]: listed object 0... 2024-05-08T15:50:15.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36159]: listed object 25... 2024-05-08T15:50:15.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36159]: saw 50 objects 2024-05-08T15:50:15.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36159]: shutting down. 2024-05-08T15:50:16.751 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:16.751 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36158]: starting. 2024-05-08T15:50:16.751 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36158]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36120 2024-05-08T15:50:16.751 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36158]: shutting down. 2024-05-08T15:50:16.752 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:16.752 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:16.752 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:16.755 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:18.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36228]: starting. 2024-05-08T15:50:18.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36228]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36227 2024-05-08T15:50:18.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36228]: created object 0... 2024-05-08T15:50:18.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36228]: created object 25... 2024-05-08T15:50:18.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36228]: created object 49... 2024-05-08T15:50:18.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36228]: finishing. 2024-05-08T15:50:18.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36228]: shutting down. 2024-05-08T15:50:19.779 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36229]: starting. 2024-05-08T15:50:19.779 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36229]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36227 2024-05-08T15:50:19.779 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36229]: shutting down. 2024-05-08T15:50:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36264]: starting. 2024-05-08T15:50:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36264]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36227 2024-05-08T15:50:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36264]: created object 0... 2024-05-08T15:50:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36264]: created object 25... 2024-05-08T15:50:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36264]: created object 49... 2024-05-08T15:50:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36264]: finishing. 2024-05-08T15:50:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36264]: shutting down. 2024-05-08T15:50:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36266]: starting. 2024-05-08T15:50:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36266]: listing objects. 2024-05-08T15:50:21.895 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36266]: listed object 0... 2024-05-08T15:50:21.895 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36266]: listed object 25... 2024-05-08T15:50:21.895 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36266]: saw 50 objects 2024-05-08T15:50:21.895 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36266]: shutting down. 2024-05-08T15:50:22.724 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:22.724 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36265]: starting. 2024-05-08T15:50:22.724 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36265]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36227 2024-05-08T15:50:22.724 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36265]: shutting down. 2024-05-08T15:50:22.725 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:22.726 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:22.726 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:22.729 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:24.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36335]: starting. 2024-05-08T15:50:24.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36335]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36334 2024-05-08T15:50:24.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36335]: created object 0... 2024-05-08T15:50:24.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36335]: created object 25... 2024-05-08T15:50:24.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36335]: created object 49... 2024-05-08T15:50:24.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36335]: finishing. 2024-05-08T15:50:24.840 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36335]: shutting down. 2024-05-08T15:50:25.736 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36336]: starting. 2024-05-08T15:50:25.737 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36336]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36334 2024-05-08T15:50:25.737 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36336]: shutting down. 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36371]: starting. 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36371]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36334 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36371]: created object 0... 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36371]: created object 25... 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36371]: created object 49... 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36371]: finishing. 2024-05-08T15:50:27.849 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36371]: shutting down. 2024-05-08T15:50:27.859 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:27.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36373]: starting. 2024-05-08T15:50:27.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36373]: listing objects. 2024-05-08T15:50:27.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36373]: listed object 0... 2024-05-08T15:50:27.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36373]: listed object 25... 2024-05-08T15:50:27.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36373]: saw 50 objects 2024-05-08T15:50:27.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36373]: shutting down. 2024-05-08T15:50:28.769 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:28.770 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36372]: starting. 2024-05-08T15:50:28.770 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36372]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36334 2024-05-08T15:50:28.770 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36372]: shutting down. 2024-05-08T15:50:28.771 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:28.771 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:28.771 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:28.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:30.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36442]: starting. 2024-05-08T15:50:30.887 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36442]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36441 2024-05-08T15:50:30.887 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36442]: created object 0... 2024-05-08T15:50:30.887 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36442]: created object 25... 2024-05-08T15:50:30.887 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36442]: created object 49... 2024-05-08T15:50:30.887 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36442]: finishing. 2024-05-08T15:50:30.887 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36442]: shutting down. 2024-05-08T15:50:31.802 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36443]: starting. 2024-05-08T15:50:31.802 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36443]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36441 2024-05-08T15:50:31.802 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36443]: shutting down. 2024-05-08T15:50:33.807 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:33.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36478]: starting. 2024-05-08T15:50:33.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36478]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36441 2024-05-08T15:50:33.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36478]: created object 0... 2024-05-08T15:50:33.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36478]: created object 25... 2024-05-08T15:50:33.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36478]: created object 49... 2024-05-08T15:50:33.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36478]: finishing. 2024-05-08T15:50:33.808 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36478]: shutting down. 2024-05-08T15:50:33.818 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:33.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36480]: starting. 2024-05-08T15:50:33.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36480]: listing objects. 2024-05-08T15:50:33.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36480]: listed object 0... 2024-05-08T15:50:33.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36480]: listed object 25... 2024-05-08T15:50:33.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36480]: saw 50 objects 2024-05-08T15:50:33.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36480]: shutting down. 2024-05-08T15:50:34.736 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:34.737 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36479]: starting. 2024-05-08T15:50:34.737 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36479]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36441 2024-05-08T15:50:34.737 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36479]: shutting down. 2024-05-08T15:50:34.739 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:34.739 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:34.739 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:34.742 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:36.834 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36549]: starting. 2024-05-08T15:50:36.834 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36549]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36548 2024-05-08T15:50:36.834 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36549]: created object 0... 2024-05-08T15:50:36.834 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36549]: created object 25... 2024-05-08T15:50:36.834 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36549]: created object 49... 2024-05-08T15:50:36.835 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36549]: finishing. 2024-05-08T15:50:36.835 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36549]: shutting down. 2024-05-08T15:50:37.764 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36550]: starting. 2024-05-08T15:50:37.764 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36550]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36548 2024-05-08T15:50:37.764 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36550]: shutting down. 2024-05-08T15:50:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36585]: starting. 2024-05-08T15:50:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36585]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36548 2024-05-08T15:50:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36585]: created object 0... 2024-05-08T15:50:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36585]: created object 25... 2024-05-08T15:50:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36585]: created object 49... 2024-05-08T15:50:39.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36585]: finishing. 2024-05-08T15:50:39.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36585]: shutting down. 2024-05-08T15:50:39.882 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:39.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36587]: starting. 2024-05-08T15:50:39.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36587]: listing objects. 2024-05-08T15:50:39.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36587]: listed object 0... 2024-05-08T15:50:39.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36587]: listed object 25... 2024-05-08T15:50:39.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36587]: saw 50 objects 2024-05-08T15:50:39.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36587]: shutting down. 2024-05-08T15:50:40.786 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:40.786 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36586]: starting. 2024-05-08T15:50:40.786 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36586]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36548 2024-05-08T15:50:40.786 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36586]: shutting down. 2024-05-08T15:50:40.787 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:40.787 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:40.787 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:40.790 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:42.816 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36656]: starting. 2024-05-08T15:50:42.816 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36656]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36655 2024-05-08T15:50:42.816 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36656]: created object 0... 2024-05-08T15:50:42.816 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36656]: created object 25... 2024-05-08T15:50:42.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36656]: created object 49... 2024-05-08T15:50:42.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36656]: finishing. 2024-05-08T15:50:42.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36656]: shutting down. 2024-05-08T15:50:43.738 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36657]: starting. 2024-05-08T15:50:43.739 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36657]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36655 2024-05-08T15:50:43.739 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36657]: shutting down. 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36692]: starting. 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36692]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36655 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36692]: created object 0... 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36692]: created object 25... 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36692]: created object 49... 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36692]: finishing. 2024-05-08T15:50:45.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36692]: shutting down. 2024-05-08T15:50:45.831 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:45.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36694]: starting. 2024-05-08T15:50:45.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36694]: listing objects. 2024-05-08T15:50:45.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36694]: listed object 0... 2024-05-08T15:50:45.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36694]: listed object 25... 2024-05-08T15:50:45.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36694]: saw 50 objects 2024-05-08T15:50:45.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36694]: shutting down. 2024-05-08T15:50:46.744 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:46.744 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36693]: starting. 2024-05-08T15:50:46.744 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36693]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36655 2024-05-08T15:50:46.745 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36693]: shutting down. 2024-05-08T15:50:46.746 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:46.746 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:46.746 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:46.749 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:48.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36763]: starting. 2024-05-08T15:50:48.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36763]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36762 2024-05-08T15:50:48.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36763]: created object 0... 2024-05-08T15:50:48.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36763]: created object 25... 2024-05-08T15:50:48.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36763]: created object 49... 2024-05-08T15:50:48.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36763]: finishing. 2024-05-08T15:50:48.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36763]: shutting down. 2024-05-08T15:50:49.774 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36764]: starting. 2024-05-08T15:50:49.774 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36764]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36762 2024-05-08T15:50:49.774 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36764]: shutting down. 2024-05-08T15:50:51.873 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:51.873 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36799]: starting. 2024-05-08T15:50:51.873 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36799]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36762 2024-05-08T15:50:51.873 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36799]: created object 0... 2024-05-08T15:50:51.873 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36799]: created object 25... 2024-05-08T15:50:51.873 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36799]: created object 49... 2024-05-08T15:50:51.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36799]: finishing. 2024-05-08T15:50:51.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36799]: shutting down. 2024-05-08T15:50:51.884 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:51.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36801]: starting. 2024-05-08T15:50:51.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36801]: listing objects. 2024-05-08T15:50:51.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36801]: listed object 0... 2024-05-08T15:50:51.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36801]: listed object 25... 2024-05-08T15:50:51.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36801]: saw 50 objects 2024-05-08T15:50:51.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36801]: shutting down. 2024-05-08T15:50:52.734 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:52.734 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36800]: starting. 2024-05-08T15:50:52.734 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36800]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36762 2024-05-08T15:50:52.734 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36800]: shutting down. 2024-05-08T15:50:52.736 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:52.736 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:52.736 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:52.740 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:50:54.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36870]: starting. 2024-05-08T15:50:54.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36870]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36869 2024-05-08T15:50:54.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36870]: created object 0... 2024-05-08T15:50:54.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36870]: created object 25... 2024-05-08T15:50:54.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36870]: created object 49... 2024-05-08T15:50:54.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36870]: finishing. 2024-05-08T15:50:54.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36870]: shutting down. 2024-05-08T15:50:55.799 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36871]: starting. 2024-05-08T15:50:55.799 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36871]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36869 2024-05-08T15:50:55.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36871]: shutting down. 2024-05-08T15:50:57.898 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:57.898 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36906]: starting. 2024-05-08T15:50:57.898 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36906]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36869 2024-05-08T15:50:57.898 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36906]: created object 0... 2024-05-08T15:50:57.898 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36906]: created object 25... 2024-05-08T15:50:57.898 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36906]: created object 49... 2024-05-08T15:50:57.899 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36906]: finishing. 2024-05-08T15:50:57.899 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[36906]: shutting down. 2024-05-08T15:50:57.909 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:57.909 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36908]: starting. 2024-05-08T15:50:57.909 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36908]: listing objects. 2024-05-08T15:50:57.909 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36908]: listed object 0... 2024-05-08T15:50:57.909 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36908]: listed object 25... 2024-05-08T15:50:57.909 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36908]: saw 50 objects 2024-05-08T15:50:57.909 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[36908]: shutting down. 2024-05-08T15:50:58.810 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:58.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36907]: starting. 2024-05-08T15:50:58.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36907]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36869 2024-05-08T15:50:58.810 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[36907]: shutting down. 2024-05-08T15:50:58.811 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:58.811 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:50:58.812 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:50:58.815 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:51:00.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36977]: starting. 2024-05-08T15:51:00.912 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36977]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36976 2024-05-08T15:51:00.912 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36977]: created object 0... 2024-05-08T15:51:00.912 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36977]: created object 25... 2024-05-08T15:51:00.912 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36977]: created object 49... 2024-05-08T15:51:00.912 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36977]: finishing. 2024-05-08T15:51:00.912 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[36977]: shutting down. 2024-05-08T15:51:01.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36978]: starting. 2024-05-08T15:51:01.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36978]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36976 2024-05-08T15:51:01.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[36978]: shutting down. 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37013]: starting. 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37013]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-36976 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37013]: created object 0... 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37013]: created object 25... 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37013]: created object 49... 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37013]: finishing. 2024-05-08T15:51:03.820 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37013]: shutting down. 2024-05-08T15:51:03.830 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:03.830 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37015]: starting. 2024-05-08T15:51:03.830 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37015]: listing objects. 2024-05-08T15:51:03.830 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37015]: listed object 0... 2024-05-08T15:51:03.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37015]: listed object 25... 2024-05-08T15:51:03.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37015]: saw 50 objects 2024-05-08T15:51:03.831 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37015]: shutting down. 2024-05-08T15:51:04.740 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:04.740 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37014]: starting. 2024-05-08T15:51:04.740 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37014]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-36976 2024-05-08T15:51:04.740 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37014]: shutting down. 2024-05-08T15:51:04.742 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:04.742 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:04.742 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:51:04.745 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:51:05.052 INFO:tasks.mon_thrash.mon_thrasher:killing mon.i 2024-05-08T15:51:05.052 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.i 2024-05-08T15:51:05.053 INFO:tasks.ceph.mon.i:Restarting daemon 2024-05-08T15:51:05.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i i 2024-05-08T15:51:05.055 INFO:tasks.ceph.mon.i:Started 2024-05-08T15:51:05.055 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T15:51:05.055 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-05-08T15:51:05.055 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T15:51:05.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T15:51:05.059 INFO:tasks.ceph.mon.c:Started 2024-05-08T15:51:05.059 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:51:05.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:51:05.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.178+0000 7ff297cc1640 10 monclient: get_monmap_and_config 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.178+0000 7ff297cc1640 10 monclient: build_initial_monmap 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.178+0000 7ff297cc1640 10 monclient: monmap: 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:05.179805+0000 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:05.179805+0000 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:05.175 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:05.176 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:05.176 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:05.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.179+0000 7ff297cc1640 1 Processor -- start 2024-05-08T15:51:05.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.179+0000 7ff297cc1640 1 -- start start 2024-05-08T15:51:05.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.179+0000 7ff297cc1640 10 monclient: init 2024-05-08T15:51:05.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 10 monclient: _add_conns ranks=[1,4,2,5,8,0,7,3,6] 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 10 monclient(hunting): picked mon.noname-f con 0x7ff2900a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 10 monclient(hunting): picked mon.noname-c con 0x7ff2900a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 10 monclient(hunting): picked mon.noname-b con 0x7ff29013e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2900a1660 con 0x7ff2900a53a0 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2900a28f0 con 0x7ff29013e310 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2900a2bd0 con 0x7ff2900a5780 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff297cc1640 10 monclient(hunting): authenticate will time out at 1073.295287s 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff29625e640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7ff29013e310 0x7ff290140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35264/0 (socket says 172.21.15.7:35264) 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff29625e640 1 -- 172.21.15.7:0/474685162 learned_addr learned my addr 172.21.15.7:0/474685162 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff295a5d640 1 --1- 172.21.15.7:0/474685162 >> v1:172.21.15.81:6789/0 conn(0x7ff2900a53a0 0x7ff2900a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:37302/0 (socket says 172.21.15.7:37302) 2024-05-08T15:51:05.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:05.180+0000 7ff29525c640 1 --1- 172.21.15.7:0/474685162 >> v1:172.21.15.7:6791/0 conn(0x7ff2900a5780 0x7ff29013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55310/0 (socket says 172.21.15.7:55310) 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient(hunting): tick 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 1 monclient(hunting): continuing hunt 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 1 -- 172.21.15.7:0/474685162 >> v1:172.21.15.7:6791/0 conn(0x7ff2900a5780 legacy=0x7ff29013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 1 -- 172.21.15.7:0/474685162 >> v1:172.21.15.7:6790/0 conn(0x7ff29013e310 legacy=0x7ff290140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 1 -- 172.21.15.7:0/474685162 >> v1:172.21.15.81:6789/0 conn(0x7ff2900a53a0 legacy=0x7ff2900a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient: _add_conns ranks=[3,0,6,8,5,7] 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient(hunting): picked mon.noname-g con 0x7ff2740037e0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:08.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient(hunting): picked mon.noname-a con 0x7ff274007470 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient(hunting): picked mon.noname-d con 0x7ff27400b200 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 1 -- 172.21.15.7:0/474685162 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2900a6730 con 0x7ff274007470 2024-05-08T15:51:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 1 -- 172.21.15.7:0/474685162 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27400ffd0 con 0x7ff2740037e0 2024-05-08T15:51:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 1 -- 172.21.15.7:0/474685162 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff274011260 con 0x7ff27400b200 2024-05-08T15:51:08.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:08.180+0000 7ff2877fe640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425444195 0 0) 0x7ff27400ffd0 con 0x7ff2740037e0 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): my global_id is 75783 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff27c003520 con 0x7ff2740037e0 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1265857207 0 0) 0x7ff2900a6730 con 0x7ff274007470 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient(hunting): my global_id is 94788 2024-05-08T15:51:10.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff27400ffd0 con 0x7ff274007470 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2598657764 0 0) 0x7ff27c003520 con 0x7ff2740037e0 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 >> v1:172.21.15.7:6792/0 conn(0x7ff27400b200 legacy=0x7ff27400d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 >> v1:172.21.15.7:6789/0 conn(0x7ff274007470 legacy=0x7ff274009930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 monclient: found mon.noname-g 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient: _send_mon_message to mon.noname-g at v1:172.21.15.81:6790/0 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740124f0 con 0x7ff2740037e0 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff29625e640 1 -- 172.21.15.7:0/474685162 reap_dead start 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff28c0017f0 con 0x7ff2740037e0 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff297cc1640 5 monclient: authenticate success, global_id 75783 2024-05-08T15:51:10.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff297cc1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient: got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff294a5b640 10 monclient: dump: 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.174 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.170+0000 7ff297cc1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 <== mon.3 v1:172.21.15.81:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff28c001a50 con 0x7ff2740037e0 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff294a5b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff294a5b640 1 -- 172.21.15.7:0/474685162 <== mon.3 v1:172.21.15.81:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff28c004b20 con 0x7ff2740037e0 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff297cc1640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff297cc1640 4 set_mon_vals no callback set 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff294a5b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff294a5b640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff294a5b640 10 monclient: dump: 2024-05-08T15:51:10.175 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.176 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff297cc1640 10 monclient: shutdown 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff297cc1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff297cc1640 1 -- 172.21.15.7:0/474685162 >> v1:172.21.15.81:6790/0 conn(0x7ff2740037e0 legacy=0x7ff274005c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff297cc1640 1 -- 172.21.15.7:0/474685162 shutdown_connections 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.171+0000 7ff297cc1640 1 -- 172.21.15.7:0/474685162 >> 172.21.15.7:0/474685162 conn(0x7ff2900a9fa0 msgr2=0x7ff2900aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.172+0000 7ff297cc1640 1 -- 172.21.15.7:0/474685162 shutdown_connections 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.172+0000 7ff297cc1640 1 -- 172.21.15.7:0/474685162 wait complete. 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.172+0000 7ff297cc1640 10 monclient: build_initial_monmap 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.172+0000 7ff297cc1640 10 monclient: monmap: 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:10.177 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.173+0000 7ff297cc1640 1 Processor -- start 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.173+0000 7ff297cc1640 1 -- start start 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.173+0000 7ff297cc1640 10 monclient: init 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.173+0000 7ff297cc1640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:10.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.173+0000 7ff297cc1640 10 monclient: _add_conns ranks=[2,8,1,5,4,6,0,3,7] 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.173+0000 7ff297cc1640 10 monclient(hunting): picked mon.noname-b con 0x7ff2740037e0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.173+0000 7ff297cc1640 10 monclient(hunting): picked mon.noname-e con 0x7ff290134180 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff297cc1640 10 monclient(hunting): picked mon.noname-f con 0x7ff2901d2fd0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff297cc1640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2900a43c0 con 0x7ff2901d2fd0 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff297cc1640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29009f6b0 con 0x7ff2740037e0 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff297cc1640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff29009e140 con 0x7ff290134180 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff297cc1640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff297cc1640 10 monclient(hunting): authenticate will time out at 1078.288756s 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff295a5d640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7ff2740037e0 0x7ff290133a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35278/0 (socket says 172.21.15.7:35278) 2024-05-08T15:51:10.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.174+0000 7ff295a5d640 1 -- 172.21.15.7:0/1411801202 learned_addr learned my addr 172.21.15.7:0/1411801202 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:10.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 310360208 0 0) 0x7ff29009f6b0 con 0x7ff2740037e0 2024-05-08T15:51:10.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): my global_id is 94916 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff274005410 con 0x7ff2740037e0 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1311525698 0 0) 0x7ff274005410 con 0x7ff2740037e0 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff29009f6b0 con 0x7ff2740037e0 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff28c001d00 con 0x7ff2740037e0 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): dump: 2024-05-08T15:51:10.210 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.211 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870752854 0 0) 0x7ff29009e140 con 0x7ff290134180 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient(hunting): my global_id is 94742 2024-05-08T15:51:10.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff274005410 con 0x7ff290134180 2024-05-08T15:51:10.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 167030966 0 0) 0x7ff29009f6b0 con 0x7ff2740037e0 2024-05-08T15:51:10.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 >> v1:172.21.15.7:6793/0 conn(0x7ff290134180 legacy=0x7ff2901d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 >> v1:172.21.15.81:6789/0 conn(0x7ff2901d2fd0 legacy=0x7ff2901d53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 monclient: found mon.b 2024-05-08T15:51:10.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:10.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff29009d3c0 con 0x7ff2740037e0 2024-05-08T15:51:10.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff2867fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff297cc1640 5 monclient: authenticate success, global_id 94916 2024-05-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff297cc1640 10 monclient: _renew_subs 2024-05-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff297cc1640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.210+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff29009ee10 con 0x7ff2740037e0 2024-05-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff297cc1640 10 monclient: _renew_subs 2024-05-08T15:51:10.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff297cc1640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7ff29009fe40 con 0x7ff2740037e0 2024-05-08T15:51:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff28c004ec0 con 0x7ff2740037e0 2024-05-08T15:51:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff2867fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff28c005430 con 0x7ff2740037e0 2024-05-08T15:51:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff2867fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff2867fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff2867fc640 10 monclient: dump: 2024-05-08T15:51:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.217 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.218 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.219 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.220 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.211+0000 7ff287fff640 4 set_mon_vals no callback set 2024-05-08T15:51:10.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.212+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7ff28c005650 con 0x7ff2740037e0 2024-05-08T15:51:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.212+0000 7ff297cc1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.212+0000 7ff297cc1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.212+0000 7ff297cc1640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.212+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(193..193 src has 1..193) ==== 3506+0+0 (unknown 2480140971 0 0) 0x7ff28c046ec0 con 0x7ff2740037e0 2024-05-08T15:51:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.212+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff290075820 con 0x7ff2740037e0 2024-05-08T15:51:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.212+0000 7ff29525c640 10 monclient: get_auth_request con 0x7ff27403b190 auth_method 0 2024-05-08T15:51:10.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.217+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7ff28c016250 con 0x7ff2740037e0 2024-05-08T15:51:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.217+0000 7ff2867fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.217+0000 7ff2867fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.482+0000 7ff297cc1640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:51:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.482+0000 7ff297cc1640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.482+0000 7ff297cc1640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.482+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff29013ad30 con 0x7ff2740037e0 2024-05-08T15:51:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.483+0000 7ff2867fc640 1 -- 172.21.15.7:0/1411801202 <== mon.2 v1:172.21.15.7:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 3322445854) 0x7ff28c015b50 con 0x7ff2740037e0 2024-05-08T15:51:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.483+0000 7ff2867fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:10.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.483+0000 7ff2867fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:10.480 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 >> v1:172.21.15.81:6812/847627118 conn(0x7ff27403b190 legacy=0x7ff27403d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 10 monclient: shutdown 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 >> v1:172.21.15.7:6790/0 conn(0x7ff2740037e0 legacy=0x7ff290133a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 shutdown_connections 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 >> 172.21.15.7:0/1411801202 conn(0x7ff2900a9fa0 msgr2=0x7ff290136950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 shutdown_connections 2024-05-08T15:51:10.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.485+0000 7ff297cc1640 1 -- 172.21.15.7:0/1411801202 wait complete. 2024-05-08T15:51:10.491 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:51:10.492 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:51:10.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.600+0000 7f8ba23b2640 10 monclient: get_monmap_and_config 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.600+0000 7f8ba23b2640 10 monclient: build_initial_monmap 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.600+0000 7f8ba23b2640 10 monclient: monmap: 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:10.601059+0000 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:10.601059+0000 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.596 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:10.597 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.600+0000 7f8ba23b2640 1 Processor -- start 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.600+0000 7f8ba23b2640 1 -- start start 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.600+0000 7f8ba23b2640 10 monclient: init 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 10 monclient: _add_conns ranks=[6,1,3,8,7,5,0,4,2] 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 10 monclient(hunting): picked mon.noname-d con 0x7f8b9c0a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 10 monclient(hunting): picked mon.noname-f con 0x7f8b9c0a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:10.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 10 monclient(hunting): picked mon.noname-g con 0x7f8b9c13e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9c0a1660 con 0x7f8b9c0a5780 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9c0a28f0 con 0x7f8b9c13e310 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9c0a2bd0 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8ba23b2640 10 monclient(hunting): authenticate will time out at 1078.716113s 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9bfff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f8b9c0a53a0 0x7f8b9c0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:46822/0 (socket says 172.21.15.7:46822) 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9bfff640 1 -- 172.21.15.7:0/65633378 learned_addr learned my addr 172.21.15.7:0/65633378 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 193988446 0 0) 0x7f8b9c0a2bd0 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9b7fe640 10 monclient(hunting): my global_id is 75642 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.601+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b88003520 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370610902 0 0) 0x7f8b9c0a28f0 con 0x7f8b9c13e310 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient(hunting): my global_id is 75792 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b9c0a2bd0 con 0x7f8b9c13e310 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1468286268 0 0) 0x7f8b88003520 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 >> v1:172.21.15.81:6790/0 conn(0x7f8b9c13e310 legacy=0x7f8b9c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 >> v1:172.21.15.81:6789/0 conn(0x7f8b9c0a5780 legacy=0x7f8b9c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 monclient: found mon.noname-d 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c0a6730 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8b8c0017b0 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: dump: 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.609 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.610 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 5 monclient: authenticate success, global_id 75642 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8b8c001a50 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 1 -- 172.21.15.7:0/65633378 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8b8c004be0 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 4 set_mon_vals no callback set 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8b9b7fe640 10 monclient: dump: 2024-05-08T15:51:10.611 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.612 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 10 monclient: shutdown 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 1 -- 172.21.15.7:0/65633378 >> v1:172.21.15.7:6792/0 conn(0x7f8b9c0a53a0 legacy=0x7f8b9c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 1 -- 172.21.15.7:0/65633378 shutdown_connections 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 1 -- 172.21.15.7:0/65633378 >> 172.21.15.7:0/65633378 conn(0x7f8b9c0a9fa0 msgr2=0x7f8b9c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 1 -- 172.21.15.7:0/65633378 shutdown_connections 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.602+0000 7f8ba23b2640 1 -- 172.21.15.7:0/65633378 wait complete. 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.603+0000 7f8ba23b2640 10 monclient: build_initial_monmap 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.603+0000 7f8ba23b2640 10 monclient: monmap: 2024-05-08T15:51:10.613 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:10.614 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.603+0000 7f8ba23b2640 1 Processor -- start 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.603+0000 7f8ba23b2640 1 -- start start 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.603+0000 7f8ba23b2640 10 monclient: init 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 10 monclient: _add_conns ranks=[3,5,8,0,4,7,6,1,2] 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 10 monclient(hunting): picked mon.noname-g con 0x7f8b9c0a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 10 monclient(hunting): picked mon.noname-h con 0x7f8b9c0a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:10.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 10 monclient(hunting): picked mon.noname-e con 0x7f8b9c13e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9c137d40 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9c0a43c0 con 0x7f8b9c0a5780 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9c0a1e00 con 0x7f8b9c13e310 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba23b2640 10 monclient(hunting): authenticate will time out at 1078.718887s 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba094f640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f8b9c13e310 0x7f8b9c1d4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50332/0 (socket says 172.21.15.7:50332) 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8ba094f640 1 -- 172.21.15.7:0/2045288760 learned_addr learned my addr 172.21.15.7:0/2045288760 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495181390 0 0) 0x7f8b9c0a1e00 con 0x7f8b9c13e310 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): my global_id is 94751 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b7c003520 con 0x7f8b9c13e310 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654355835 0 0) 0x7f8b9c137d40 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): my global_id is 75801 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b9c0a1e00 con 0x7f8b9c0a53a0 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1010257919 0 0) 0x7f8b9c0a43c0 con 0x7f8b9c0a5780 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): my global_id is 5594 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b9c137d40 con 0x7f8b9c0a5780 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1948167359 0 0) 0x7f8b7c003520 con 0x7f8b9c13e310 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8b9c0a43c0 con 0x7f8b9c13e310 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8b94001db0 con 0x7f8b9c13e310 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): dump: 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.618 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:10.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.604+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4131388081 0 0) 0x7f8b9c0a43c0 con 0x7f8b9c13e310 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.81:6791/0 conn(0x7f8b9c0a5780 legacy=0x7f8b9c134490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.81:6790/0 conn(0x7f8b9c0a53a0 legacy=0x7f8b9c133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 1 monclient: found mon.e 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c0a0870 con 0x7f8b9c13e310 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 5 monclient: authenticate success, global_id 94751 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 10 monclient: _renew_subs 2024-05-08T15:51:10.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c134ba0 con 0x7f8b9c13e310 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 10 monclient: _renew_subs 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f8b9c136330 con 0x7f8b9c13e310 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8b940020b0 con 0x7f8b9c13e310 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8b94003f80 con 0x7f8b9c13e310 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b9affd640 4 set_mon_vals no callback set 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 10 monclient: dump: 2024-05-08T15:51:10.621 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.622 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8b800065c0 con 0x7f8b80002860 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.605+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f8b94004120 con 0x7f8b9c13e310 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(193..193 src has 1..193) ==== 3506+0+0 (unknown 2480140971 0 0) 0x7f8b94046f50 con 0x7f8b9c13e310 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 70570718 0 0) 0x7f8b800065c0 con 0x7f8b80002860 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 10 monclient(hunting): my global_id is 94797 2024-05-08T15:51:10.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b9c0a43c0 con 0x7f8b80002860 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b93fff640 10 monclient: get_auth_request con 0x7f8b7c0398a0 auth_method 0 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 249381012 0 0) 0x7f8b9c0a43c0 con 0x7f8b80002860 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8b800065c0 con 0x7f8b80002860 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8b8c001160 con 0x7f8b80002860 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 10 monclient: dump: 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.624 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1682816203 0 0) 0x7f8b800065c0 con 0x7f8b80002860 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.606+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8b8000a150 con 0x7f8b80002860 2024-05-08T15:51:10.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.607+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f8b8c005090 con 0x7f8b80002860 2024-05-08T15:51:10.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.607+0000 7f8b997fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.607+0000 7f8b997fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:10.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.607+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.7:6789/0 conn(0x7f8b80002860 legacy=0x7f8b80004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.607+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.7:6789/0 conn(0x7f8b80002860 legacy=0x7f8b80004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8ba23b2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8b80009e20 con 0x7f8b800090e0 2024-05-08T15:51:10.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778852616 0 0) 0x7f8b80009e20 con 0x7f8b800090e0 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8b997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8b997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8b997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8b997fa640 10 monclient(hunting): my global_id is 94806 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.613+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b800065c0 con 0x7f8b800090e0 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1727024572 0 0) 0x7f8b800065c0 con 0x7f8b800090e0 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8b80009e20 con 0x7f8b800090e0 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8b8c0057f0 con 0x7f8b800090e0 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 10 monclient: dump: 2024-05-08T15:51:10.628 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.629 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1121739217 0 0) 0x7f8b80009e20 con 0x7f8b800090e0 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8b8000c390 con 0x7f8b800090e0 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 2695040052) 0x7f8b8c007ab0 con 0x7f8b800090e0 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.7:6789/0 conn(0x7f8b800090e0 legacy=0x7f8b80009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8b997fa640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.7:6789/0 conn(0x7f8b800090e0 legacy=0x7f8b80009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:10.630 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 121165, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:10.631 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.633 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:10.634 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:10.635 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.636 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:10.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:10.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.639 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:10.647 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.648 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:10.650 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.651 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:10.652 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:10.652 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:10.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.652 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:10.652 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:10.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.81:6812/847627118 conn(0x7f8b7c0398a0 legacy=0x7f8b7c03bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba23b2640 10 monclient: shutdown 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba23b2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 >> v1:172.21.15.7:6793/0 conn(0x7f8b9c13e310 legacy=0x7f8b9c1d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba094f640 1 -- 172.21.15.7:0/2045288760 reap_dead start 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba094f640 1 -- 172.21.15.7:0/2045288760 reap_dead start 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 shutdown_connections 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 >> 172.21.15.7:0/2045288760 conn(0x7f8b9c0a9fa0 msgr2=0x7f8b9c0a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.614+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 shutdown_connections 2024-05-08T15:51:10.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.615+0000 7f8ba23b2640 1 -- 172.21.15.7:0/2045288760 wait complete. 2024-05-08T15:51:10.655 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:10.655 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:51:10.655 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:51:10.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.762+0000 7f790cb98640 10 monclient: get_monmap_and_config 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.762+0000 7f790cb98640 10 monclient: build_initial_monmap 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.762+0000 7f790cb98640 10 monclient: monmap: 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:10.763790+0000 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:10.763790+0000 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:10.760 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:10.760 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:10.760 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:10.760 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:10.760 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:10.760 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:10.760 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.763+0000 7f790cb98640 1 Processor -- start 2024-05-08T15:51:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.763+0000 7f790cb98640 1 -- start start 2024-05-08T15:51:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.763+0000 7f790cb98640 10 monclient: init 2024-05-08T15:51:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 10 monclient: _add_conns ranks=[6,0,8,7,4,3,2,1,5] 2024-05-08T15:51:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 10 monclient(hunting): picked mon.noname-d con 0x7f7908137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 10 monclient(hunting): picked mon.noname-a con 0x7f79080a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 10 monclient(hunting): picked mon.noname-e con 0x7f79080a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79080a2c70 con 0x7f79080a49a0 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79080a2f50 con 0x7f7908137940 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79080a1670 con 0x7f79080a41b0 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 10 monclient(hunting): authenticate will time out at 1078.878796s 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f7905d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f79080a49a0 0x7f790813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49710/0 (socket says 172.21.15.7:49710) 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790659c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f7908137940 0x7f7908137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:46830/0 (socket says 172.21.15.7:46830) 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f7905d9b640 1 -- 172.21.15.7:0/544474767 learned_addr learned my addr 172.21.15.7:0/544474767 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 565322597 0 0) 0x7f79080a2c70 con 0x7f79080a49a0 2024-05-08T15:51:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): my global_id is 94815 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78f0003520 con 0x7f79080a49a0 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962134167 0 0) 0x7f79080a1670 con 0x7f79080a41b0 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): my global_id is 94760 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79080a2c70 con 0x7f79080a41b0 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3263280318 0 0) 0x7f79080a2f50 con 0x7f7908137940 2024-05-08T15:51:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient(hunting): my global_id is 75651 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79080a1670 con 0x7f7908137940 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1715281965 0 0) 0x7f78f0003520 con 0x7f79080a49a0 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 >> v1:172.21.15.7:6793/0 conn(0x7f79080a41b0 legacy=0x7f7908140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 >> v1:172.21.15.7:6792/0 conn(0x7f7908137940 legacy=0x7f7908137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 monclient: found mon.noname-a 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79080a39f0 con 0x7f79080a49a0 2024-05-08T15:51:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f78fc0017b0 con 0x7f79080a49a0 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790559a640 10 monclient: dump: 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.773 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.764+0000 7f790cb98640 5 monclient: authenticate success, global_id 94815 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f78fc001ad0 con 0x7f79080a49a0 2024-05-08T15:51:10.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790559a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790559a640 1 -- 172.21.15.7:0/544474767 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f78fc001d40 con 0x7f79080a49a0 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 4 set_mon_vals no callback set 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790559a640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790559a640 10 monclient: dump: 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.775 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 10 monclient: shutdown 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 1 -- 172.21.15.7:0/544474767 >> v1:172.21.15.7:6789/0 conn(0x7f79080a49a0 legacy=0x7f790813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 1 -- 172.21.15.7:0/544474767 shutdown_connections 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 1 -- 172.21.15.7:0/544474767 >> 172.21.15.7:0/544474767 conn(0x7f79080a9fa0 msgr2=0x7f79080aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 1 -- 172.21.15.7:0/544474767 shutdown_connections 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.765+0000 7f790cb98640 1 -- 172.21.15.7:0/544474767 wait complete. 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.766+0000 7f790cb98640 10 monclient: build_initial_monmap 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.766+0000 7f790cb98640 10 monclient: monmap: 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:10.777 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.766+0000 7f790cb98640 1 Processor -- start 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.766+0000 7f790cb98640 1 -- start start 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.766+0000 7f790cb98640 10 monclient: init 2024-05-08T15:51:10.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient: _add_conns ranks=[1,7,6,4,2,8,0,3,5] 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient(hunting): picked mon.noname-f con 0x7f79080a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient(hunting): picked mon.noname-i con 0x7f79080a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient(hunting): picked mon.noname-d con 0x7f7908137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7908141880 con 0x7f79080a41b0 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7908138b30 con 0x7f7908137940 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7908135c10 con 0x7f79080a49a0 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient(hunting): authenticate will time out at 1078.881799s 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f7906d9d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f7908137940 0x7f79081d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:46846/0 (socket says 172.21.15.7:46846) 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f7906d9d640 1 -- 172.21.15.7:0/1982624469 learned_addr learned my addr 172.21.15.7:0/1982624469 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:10.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3558957280 0 0) 0x7f7908138b30 con 0x7f7908137940 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): my global_id is 75660 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78e8003520 con 0x7f7908137940 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2032942550 0 0) 0x7f78e8003520 con 0x7f7908137940 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7908138b30 con 0x7f7908137940 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79000017f0 con 0x7f7908137940 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): dump: 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.780 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.781 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4074019676 0 0) 0x7f7908141880 con 0x7f79080a41b0 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient(hunting): my global_id is 5572 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78e8003520 con 0x7f79080a41b0 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2622928314 0 0) 0x7f7908138b30 con 0x7f7908137940 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.81:6792/0 conn(0x7f79080a49a0 legacy=0x7f79081344e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.81:6789/0 conn(0x7f79080a41b0 legacy=0x7f7908133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 monclient: found mon.d 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7908134bf0 con 0x7f7908137940 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f78f6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 5 monclient: authenticate success, global_id 75660 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient: _renew_subs 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.767+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7908137470 con 0x7f7908137940 2024-05-08T15:51:10.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f790cb98640 10 monclient: _renew_subs 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f790cb98640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f79081367b0 con 0x7f7908137940 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7900003ca0 con 0x7f7908137940 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f78f6ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7900004210 con 0x7f7908137940 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f78f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f7904d99640 4 set_mon_vals no callback set 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f78f6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f78f6ffd640 10 monclient: dump: 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.784 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.785 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.768+0000 7f790cb98640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.769+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f78ec0065c0 con 0x7f78ec002860 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.769+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7900004430 con 0x7f7908137940 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.769+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(193..193 src has 1..193) ==== 3506+0+0 (unknown 2480140971 0 0) 0x7f7900047280 con 0x7f7908137940 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.769+0000 7f7905d9b640 10 monclient: get_auth_request con 0x7f78e8039980 auth_method 0 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809330044 0 0) 0x7f78ec0065c0 con 0x7f78ec002860 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient(hunting): my global_id is 94925 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7908138b30 con 0x7f78ec002860 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1258957699 0 0) 0x7f7908138b30 con 0x7f78ec002860 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f78ec0065c0 con 0x7f78ec002860 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f78f80016a0 con 0x7f78ec002860 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:10.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient: dump: 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.788 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 855568353 0 0) 0x7f78ec0065c0 con 0x7f78ec002860 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78ec00a150 con 0x7f78ec002860 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f78f8001ad0 con 0x7f78ec002860 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.7:6790/0 conn(0x7f78ec002860 legacy=0x7f78ec004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.772+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.7:6790/0 conn(0x7f78ec002860 legacy=0x7f78ec004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f790cb98640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f78ec009e20 con 0x7f78ec0090e0 2024-05-08T15:51:10.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502862446 0 0) 0x7f78ec009e20 con 0x7f78ec0090e0 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f78f6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f78f6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f78f6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f78f6ffd640 10 monclient(hunting): my global_id is 94934 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78ec0065c0 con 0x7f78ec0090e0 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.779+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3390305829 0 0) 0x7f78ec0065c0 con 0x7f78ec0090e0 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f78ec009e20 con 0x7f78ec0090e0 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f78f8004ba0 con 0x7f78ec0090e0 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 10 monclient: dump: 2024-05-08T15:51:10.792 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.793 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3435958798 0 0) 0x7f78ec009e20 con 0x7f78ec0090e0 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f78ec00c390 con 0x7f78ec0090e0 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 2109981860) 0x7f78f8004ea0 con 0x7f78ec0090e0 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.7:6790/0 conn(0x7f78ec0090e0 legacy=0x7f78ec009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f78f6ffd640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.7:6790/0 conn(0x7f78ec0090e0 legacy=0x7f78ec009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:10.794 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 121331, 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:10.795 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.797 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.798 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:10.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.800 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:10.803 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:10.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.806 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:10.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.813 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:10.815 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.81:6812/847627118 conn(0x7f78e8039980 legacy=0x7f78e803be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 10 monclient: shutdown 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 >> v1:172.21.15.7:6792/0 conn(0x7f7908137940 legacy=0x7f79081d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f7906d9d640 1 -- 172.21.15.7:0/1982624469 reap_dead start 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f7906d9d640 1 -- 172.21.15.7:0/1982624469 reap_dead start 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 shutdown_connections 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 >> 172.21.15.7:0/1982624469 conn(0x7f79080a9fa0 msgr2=0x7f7908130760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 shutdown_connections 2024-05-08T15:51:10.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.780+0000 7f790cb98640 1 -- 172.21.15.7:0/1982624469 wait complete. 2024-05-08T15:51:10.818 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:10.818 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:51:10.818 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:51:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.926+0000 7f0c3aab3640 10 monclient: get_monmap_and_config 2024-05-08T15:51:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.926+0000 7f0c3aab3640 10 monclient: build_initial_monmap 2024-05-08T15:51:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.926+0000 7f0c3aab3640 10 monclient: monmap: 2024-05-08T15:51:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:10.927485+0000 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:10.927485+0000 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:10.923 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 1 Processor -- start 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 1 -- start start 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient: init 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient: _add_conns ranks=[0,4,7,2,1,5,8,3,6] 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient(hunting): picked mon.noname-a con 0x7f0c34137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient(hunting): picked mon.noname-c con 0x7f0c340a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient(hunting): picked mon.noname-i con 0x7f0c340a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340a2c70 con 0x7f0c34137940 2024-05-08T15:51:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340a2f50 con 0x7f0c340a49a0 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340a1670 con 0x7f0c340a41b0 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3aab3640 10 monclient(hunting): authenticate will time out at 1079.042461s 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3884f640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f0c34137940 0x7f0c34137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49714/0 (socket says 172.21.15.7:49714) 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c33fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f0c340a49a0 0x7f0c3413cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55318/0 (socket says 172.21.15.7:55318) 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.927+0000 7f0c3884f640 1 -- 172.21.15.7:0/224167306 learned_addr learned my addr 172.21.15.7:0/224167306 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059628169 0 0) 0x7f0c340a2f50 con 0x7f0c340a49a0 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): my global_id is 114106 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c20003520 con 0x7f0c340a49a0 2024-05-08T15:51:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824261268 0 0) 0x7f0c340a2c70 con 0x7f0c34137940 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient(hunting): my global_id is 94824 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c340a2f50 con 0x7f0c34137940 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2366665803 0 0) 0x7f0c20003520 con 0x7f0c340a49a0 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 >> v1:172.21.15.81:6792/0 conn(0x7f0c340a41b0 legacy=0x7f0c34140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 >> v1:172.21.15.7:6789/0 conn(0x7f0c34137940 legacy=0x7f0c34137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 monclient: found mon.noname-c 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c340a39f0 con 0x7f0c340a49a0 2024-05-08T15:51:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0c1c0017b0 con 0x7f0c340a49a0 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c3aab3640 5 monclient: authenticate success, global_id 114106 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c3aab3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: dump: 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c3aab3640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0c1c001a90 con 0x7f0c340a49a0 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 1 -- 172.21.15.7:0/224167306 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0c1c004b20 con 0x7f0c340a49a0 2024-05-08T15:51:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c3aab3640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c3aab3640 4 set_mon_vals no callback set 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.928+0000 7f0c337fe640 10 monclient: dump: 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 10 monclient: shutdown 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 1 -- 172.21.15.7:0/224167306 >> v1:172.21.15.7:6791/0 conn(0x7f0c340a49a0 legacy=0x7f0c3413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 1 -- 172.21.15.7:0/224167306 shutdown_connections 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 1 -- 172.21.15.7:0/224167306 >> 172.21.15.7:0/224167306 conn(0x7f0c340a9fa0 msgr2=0x7f0c340aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 1 -- 172.21.15.7:0/224167306 shutdown_connections 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 1 -- 172.21.15.7:0/224167306 wait complete. 2024-05-08T15:51:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 10 monclient: build_initial_monmap 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.929+0000 7f0c3aab3640 10 monclient: monmap: 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 1 Processor -- start 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 1 -- start start 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient: init 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient: _add_conns ranks=[4,8,3,1,0,2,5,7,6] 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient(hunting): picked mon.noname-c con 0x7f0c340a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient(hunting): picked mon.noname-e con 0x7f0c340a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient(hunting): picked mon.noname-g con 0x7f0c34137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c34135bf0 con 0x7f0c34137940 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c340a0f60 con 0x7f0c340a41b0 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c34133b00 con 0x7f0c340a49a0 2024-05-08T15:51:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3aab3640 10 monclient(hunting): authenticate will time out at 1079.045520s 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3884f640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f0c340a41b0 0x7f0c34136010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55330/0 (socket says 172.21.15.7:55330) 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.930+0000 7f0c3884f640 1 -- 172.21.15.7:0/2451659195 learned_addr learned my addr 172.21.15.7:0/2451659195 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553672003 0 0) 0x7f0c340a0f60 con 0x7f0c340a41b0 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): my global_id is 114115 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c14003520 con 0x7f0c340a41b0 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792744114 0 0) 0x7f0c34133b00 con 0x7f0c340a49a0 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): my global_id is 94769 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c340a0f60 con 0x7f0c340a49a0 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3572825344 0 0) 0x7f0c14003520 con 0x7f0c340a41b0 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0c34133b00 con 0x7f0c340a41b0 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 124672959 0 0) 0x7f0c34135bf0 con 0x7f0c34137940 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): my global_id is 75810 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c14003520 con 0x7f0c34137940 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0c24001d00 con 0x7f0c340a41b0 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:10.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): dump: 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.943 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2346876992 0 0) 0x7f0c340a0f60 con 0x7f0c340a49a0 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0c34135bf0 con 0x7f0c340a49a0 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0c1c004b20 con 0x7f0c340a49a0 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient(hunting): dump: 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.944 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 388950056 0 0) 0x7f0c34133b00 con 0x7f0c340a41b0 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.7:6793/0 conn(0x7f0c340a49a0 legacy=0x7f0c341d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.81:6790/0 conn(0x7f0c34137940 legacy=0x7f0c341d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: _finish_hunting 0 2024-05-08T15:51:10.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 monclient: found mon.c 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34135350 con 0x7f0c340a41b0 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: _finish_auth 0 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: _check_auth_tickets 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c3aab3640 5 monclient: authenticate success, global_id 114115 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c3aab3640 10 monclient: _renew_subs 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c3aab3640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0c34135630 con 0x7f0c340a41b0 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c3aab3640 10 monclient: _renew_subs 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c3aab3640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f0c341d5a90 con 0x7f0c340a41b0 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0c24001fc0 con 0x7f0c340a41b0 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0c24005150 con 0x7f0c340a41b0 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c317fa640 10 monclient: dump: 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.931+0000 7f0c32ffd640 4 set_mon_vals no callback set 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c3aab3640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0c180065c0 con 0x7f0c18002860 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f0c240104d0 con 0x7f0c340a41b0 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(193..193 src has 1..193) ==== 3506+0+0 (unknown 2480140971 0 0) 0x7f0c24047040 con 0x7f0c340a41b0 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1748089205 0 0) 0x7f0c180065c0 con 0x7f0c18002860 2024-05-08T15:51:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 10 monclient(hunting): my global_id is 114124 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.932+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c34133b00 con 0x7f0c18002860 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c39050640 10 monclient: get_auth_request con 0x7f0c1403a2b0 auth_method 0 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3937396048 0 0) 0x7f0c34133b00 con 0x7f0c18002860 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0c180065c0 con 0x7f0c18002860 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0c1c004e20 con 0x7f0c18002860 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 10 monclient: dump: 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1731012700 0 0) 0x7f0c180065c0 con 0x7f0c18002860 2024-05-08T15:51:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c1800a150 con 0x7f0c18002860 2024-05-08T15:51:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0c1c007480 con 0x7f0c18002860 2024-05-08T15:51:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.7:6791/0 conn(0x7f0c18002860 legacy=0x7f0c18004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.933+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.7:6791/0 conn(0x7f0c18002860 legacy=0x7f0c18004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c3aab3640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0c18009e20 con 0x7f0c180090e0 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735237673 0 0) 0x7f0c18009e20 con 0x7f0c180090e0 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient(hunting): my global_id is 114133 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c180065c0 con 0x7f0c180090e0 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3673007063 0 0) 0x7f0c180065c0 con 0x7f0c180090e0 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0c18009e20 con 0x7f0c180090e0 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0c1c005600 con 0x7f0c180090e0 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient: dump: 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:10.953 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:10.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3311921639 0 0) 0x7f0c18009e20 con 0x7f0c180090e0 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0c1800c390 con 0x7f0c180090e0 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 274409611) 0x7f0c1c007630 con 0x7f0c180090e0 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.7:6791/0 conn(0x7f0c180090e0 legacy=0x7f0c18009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.940+0000 7f0c317fa640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.7:6791/0 conn(0x7f0c180090e0 legacy=0x7f0c18009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:10.955 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 5784, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:10.956 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.957 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:10.959 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:10.960 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.962 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:10.963 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.964 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:10.965 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.966 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.967 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.970 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.971 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.972 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:10.973 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.81:6812/847627118 conn(0x7f0c1403a2b0 legacy=0x7f0c1403c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 10 monclient: shutdown 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 >> v1:172.21.15.7:6791/0 conn(0x7f0c340a41b0 legacy=0x7f0c34136010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c39050640 1 -- 172.21.15.7:0/2451659195 reap_dead start 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c39050640 1 -- 172.21.15.7:0/2451659195 reap_dead start 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 shutdown_connections 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 >> 172.21.15.7:0/2451659195 conn(0x7f0c340a9fa0 msgr2=0x7f0c34130bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 shutdown_connections 2024-05-08T15:51:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:10.941+0000 7f0c3aab3640 1 -- 172.21.15.7:0/2451659195 wait complete. 2024-05-08T15:51:10.976 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:10.976 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:51:10.976 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:51:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.083+0000 7f5e98a43640 10 monclient: get_monmap_and_config 2024-05-08T15:51:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.083+0000 7f5e98a43640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.083+0000 7f5e98a43640 10 monclient: monmap: 2024-05-08T15:51:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:11.084280+0000 2024-05-08T15:51:11.079 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:11.084280+0000 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.080 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 1 Processor -- start 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 1 -- start start 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient: init 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient: _add_conns ranks=[8,3,0,1,7,2,4,5,6] 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient(hunting): picked mon.noname-e con 0x7f5e940a53a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient(hunting): picked mon.noname-g con 0x7f5e940a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient(hunting): picked mon.noname-a con 0x7f5e9413e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e940a1660 con 0x7f5e9413e310 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e940a28f0 con 0x7f5e940a5780 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e940a2bd0 con 0x7f5e940a53a0 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e98a43640 10 monclient(hunting): authenticate will time out at 1079.199285s 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e92d9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f5e9413e310 0x7f5e94140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49718/0 (socket says 172.21.15.7:49718) 2024-05-08T15:51:11.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e9259c640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f5e940a53a0 0x7f5e940a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50370/0 (socket says 172.21.15.7:50370) 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.084+0000 7f5e92d9d640 1 -- 172.21.15.7:0/2671009684 learned_addr learned my addr 172.21.15.7:0/2671009684 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780616301 0 0) 0x7f5e940a2bd0 con 0x7f5e940a53a0 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): my global_id is 94778 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e80003520 con 0x7f5e940a53a0 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2763068423 0 0) 0x7f5e940a1660 con 0x7f5e9413e310 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): my global_id is 94833 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e940a2bd0 con 0x7f5e9413e310 2024-05-08T15:51:11.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3946770986 0 0) 0x7f5e940a28f0 con 0x7f5e940a5780 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient(hunting): my global_id is 75819 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e940a1660 con 0x7f5e940a5780 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1181076952 0 0) 0x7f5e80003520 con 0x7f5e940a53a0 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 >> v1:172.21.15.81:6790/0 conn(0x7f5e940a5780 legacy=0x7f5e9413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 >> v1:172.21.15.7:6789/0 conn(0x7f5e9413e310 legacy=0x7f5e94140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 monclient: found mon.noname-e 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940a6730 con 0x7f5e940a53a0 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e840016e0 con 0x7f5e940a53a0 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: dump: 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.092 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 5 monclient: authenticate success, global_id 94778 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5e84001940 con 0x7f5e940a53a0 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 1 -- 172.21.15.7:0/2671009684 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e84004b20 con 0x7f5e940a53a0 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 4 set_mon_vals no callback set 2024-05-08T15:51:11.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e91d9b640 10 monclient: dump: 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.094 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 10 monclient: shutdown 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 1 -- 172.21.15.7:0/2671009684 >> v1:172.21.15.7:6793/0 conn(0x7f5e940a53a0 legacy=0x7f5e940a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 1 -- 172.21.15.7:0/2671009684 shutdown_connections 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.085+0000 7f5e98a43640 1 -- 172.21.15.7:0/2671009684 >> 172.21.15.7:0/2671009684 conn(0x7f5e940a9fa0 msgr2=0x7f5e940aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.086+0000 7f5e98a43640 1 -- 172.21.15.7:0/2671009684 shutdown_connections 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.086+0000 7f5e98a43640 1 -- 172.21.15.7:0/2671009684 wait complete. 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.086+0000 7f5e98a43640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.086+0000 7f5e98a43640 10 monclient: monmap: 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:11.095 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 1 Processor -- start 2024-05-08T15:51:11.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 1 -- start start 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient: init 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient: _add_conns ranks=[4,6,2,1,3,7,8,5,0] 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient(hunting): picked mon.noname-c con 0x7f5e940a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient(hunting): picked mon.noname-d con 0x7f5e940a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient(hunting): picked mon.noname-b con 0x7f5e9413e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e940a43c0 con 0x7f5e9413e310 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e94132520 con 0x7f5e940a53a0 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e941357a0 con 0x7f5e940a5780 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e98a43640 10 monclient(hunting): authenticate will time out at 1079.202159s 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8bfff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f5e940a5780 0x7f5e94131700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:46860/0 (socket says 172.21.15.7:46860) 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8bfff640 1 -- 172.21.15.7:0/3967539125 learned_addr learned my addr 172.21.15.7:0/3967539125 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3977090829 0 0) 0x7f5e8c001550 con 0x7f5e9413e310 2024-05-08T15:51:11.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): my global_id is 94943 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e74003520 con 0x7f5e9413e310 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3735553038 0 0) 0x7f5e94132520 con 0x7f5e940a53a0 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): my global_id is 114142 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e8c001550 con 0x7f5e940a53a0 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 872040502 0 0) 0x7f5e941357a0 con 0x7f5e940a5780 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 10 monclient(hunting): my global_id is 75669 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.087+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e94132520 con 0x7f5e940a5780 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2768491003 0 0) 0x7f5e8c001550 con 0x7f5e940a53a0 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5e941357a0 con 0x7f5e940a53a0 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2543976849 0 0) 0x7f5e74003520 con 0x7f5e9413e310 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5e8c001550 con 0x7f5e9413e310 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e84004b20 con 0x7f5e940a53a0 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): dump: 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.099 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3778165186 0 0) 0x7f5e94132520 con 0x7f5e940a5780 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5e74003520 con 0x7f5e940a5780 2024-05-08T15:51:11.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e8c001ec0 con 0x7f5e9413e310 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): dump: 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.101 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.102 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e7c001a50 con 0x7f5e940a5780 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient(hunting): dump: 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.103 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1248635450 0 0) 0x7f5e8c001550 con 0x7f5e9413e310 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.7:6792/0 conn(0x7f5e940a5780 legacy=0x7f5e94131700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.7:6791/0 conn(0x7f5e940a53a0 legacy=0x7f5e94138220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 monclient: found mon.b 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e941371f0 con 0x7f5e9413e310 2024-05-08T15:51:11.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e98a43640 5 monclient: authenticate success, global_id 94943 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e98a43640 10 monclient: _renew_subs 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e98a43640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5e94136490 con 0x7f5e9413e310 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e98a43640 10 monclient: _renew_subs 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e98a43640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5e941d5af0 con 0x7f5e9413e310 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5e8c004e10 con 0x7f5e9413e310 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e8c005300 con 0x7f5e9413e310 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e9159a640 4 set_mon_vals no callback set 2024-05-08T15:51:11.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.088+0000 7f5e8affd640 10 monclient: dump: 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.106 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e98a43640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5e780065c0 con 0x7f5e78002860 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f5e8c010680 con 0x7f5e9413e310 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(193..193 src has 1..193) ==== 3506+0+0 (unknown 2480140971 0 0) 0x7f5e8c047040 con 0x7f5e9413e310 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645269158 0 0) 0x7f5e780065c0 con 0x7f5e78002860 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 10 monclient(hunting): my global_id is 75678 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e8c001550 con 0x7f5e78002860 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8bfff640 10 monclient: get_auth_request con 0x7f5e7403a3f0 auth_method 0 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1824944072 0 0) 0x7f5e8c001550 con 0x7f5e78002860 2024-05-08T15:51:11.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5e780065c0 con 0x7f5e78002860 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.089+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e84001160 con 0x7f5e78002860 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 10 monclient: dump: 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.108 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3040901963 0 0) 0x7f5e780065c0 con 0x7f5e78002860 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e7800a150 con 0x7f5e78002860 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5e840071d0 con 0x7f5e78002860 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.7:6792/0 conn(0x7f5e78002860 legacy=0x7f5e78004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.090+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.7:6792/0 conn(0x7f5e78002860 legacy=0x7f5e78004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.096+0000 7f5e98a43640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.096+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5e78009e20 con 0x7f5e780090e0 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 55952872 0 0) 0x7f5e78009e20 con 0x7f5e780090e0 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient(hunting): my global_id is 75687 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e780065c0 con 0x7f5e780090e0 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3125205905 0 0) 0x7f5e780065c0 con 0x7f5e780090e0 2024-05-08T15:51:11.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5e78009e20 con 0x7f5e780090e0 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5e840058a0 con 0x7f5e780090e0 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient: dump: 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.112 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3998855250 0 0) 0x7f5e78009e20 con 0x7f5e780090e0 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5e7800c390 con 0x7f5e780090e0 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7863 (unknown 0 0 74513744) 0x7f5e84006c20 con 0x7f5e780090e0 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.7:6792/0 conn(0x7f5e780090e0 legacy=0x7f5e78009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.097+0000 7f5e8affd640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.7:6792/0 conn(0x7f5e780090e0 legacy=0x7f5e78009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.113 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 263411, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:11.114 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.115 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.116 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:11.117 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.118 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.119 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.120 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.121 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.122 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.123 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.124 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.125 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.126 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.127 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.128 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:11.129 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.130 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.131 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:11.132 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.81:6812/847627118 conn(0x7f5e7403a3f0 legacy=0x7f5e7403c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 10 monclient: shutdown 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 >> v1:172.21.15.7:6790/0 conn(0x7f5e9413e310 legacy=0x7f5e94131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e92d9d640 1 -- 172.21.15.7:0/3967539125 reap_dead start 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e92d9d640 1 -- 172.21.15.7:0/3967539125 reap_dead start 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 shutdown_connections 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 >> 172.21.15.7:0/3967539125 conn(0x7f5e940a9fa0 msgr2=0x7f5e940a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 shutdown_connections 2024-05-08T15:51:11.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.098+0000 7f5e98a43640 1 -- 172.21.15.7:0/3967539125 wait complete. 2024-05-08T15:51:11.135 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:11.135 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:51:11.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:51:11.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.249+0000 7fba9ddfb640 10 monclient: get_monmap_and_config 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.249+0000 7fba9ddfb640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.249+0000 7fba9ddfb640 10 monclient: monmap: 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:11.250792+0000 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:11.250792+0000 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.246 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.247 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.247 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.247 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.247 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.250+0000 7fba9ddfb640 1 Processor -- start 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.250+0000 7fba9ddfb640 1 -- start start 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.250+0000 7fba9ddfb640 10 monclient: init 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 10 monclient: _add_conns ranks=[4,2,0,1,3,5,7,8,6] 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 10 monclient(hunting): picked mon.noname-c con 0x7fba98137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 10 monclient(hunting): picked mon.noname-b con 0x7fba980a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 10 monclient(hunting): picked mon.noname-a con 0x7fba980a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980a2c70 con 0x7fba980a41b0 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980a2f50 con 0x7fba980a49a0 2024-05-08T15:51:11.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980a1670 con 0x7fba98137940 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba977fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fba98137940 0x7fba98137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55360/0 (socket says 172.21.15.7:55360) 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba977fe640 1 -- 172.21.15.7:0/1878221362 learned_addr learned my addr 172.21.15.7:0/1878221362 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 10 monclient(hunting): authenticate will time out at 1079.365943s 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba96ffd640 1 --1- 172.21.15.7:0/1878221362 >> v1:172.21.15.7:6790/0 conn(0x7fba980a49a0 0x7fba9813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35324/0 (socket says 172.21.15.7:35324) 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2698295045 0 0) 0x7fba980a1670 con 0x7fba98137940 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): my global_id is 114151 2024-05-08T15:51:11.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba80003520 con 0x7fba98137940 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 204110925 0 0) 0x7fba980a2c70 con 0x7fba980a41b0 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): my global_id is 94842 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba980a1670 con 0x7fba980a41b0 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 606452284 0 0) 0x7fba980a2f50 con 0x7fba980a49a0 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient(hunting): my global_id is 94952 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba980a2c70 con 0x7fba980a49a0 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 204191373 0 0) 0x7fba980a1670 con 0x7fba980a41b0 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 >> v1:172.21.15.7:6791/0 conn(0x7fba98137940 legacy=0x7fba98137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 >> v1:172.21.15.7:6790/0 conn(0x7fba980a49a0 legacy=0x7fba9813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 monclient: found mon.noname-a 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba980a39f0 con 0x7fba980a41b0 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fba90001b10 con 0x7fba980a41b0 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.251+0000 7fba9ddfb640 5 monclient: authenticate success, global_id 94842 2024-05-08T15:51:11.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 10 monclient: dump: 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.260 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fba90001d70 con 0x7fba980a41b0 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 1 -- 172.21.15.7:0/1878221362 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fba90003cd0 con 0x7fba980a41b0 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 4 set_mon_vals no callback set 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba967fc640 10 monclient: dump: 2024-05-08T15:51:11.261 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.262 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 10 monclient: shutdown 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 1 -- 172.21.15.7:0/1878221362 >> v1:172.21.15.7:6789/0 conn(0x7fba980a41b0 legacy=0x7fba98140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 1 -- 172.21.15.7:0/1878221362 shutdown_connections 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 1 -- 172.21.15.7:0/1878221362 >> 172.21.15.7:0/1878221362 conn(0x7fba980a9fa0 msgr2=0x7fba980aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 1 -- 172.21.15.7:0/1878221362 shutdown_connections 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.252+0000 7fba9ddfb640 1 -- 172.21.15.7:0/1878221362 wait complete. 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.253+0000 7fba9ddfb640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.253+0000 7fba9ddfb640 10 monclient: monmap: 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:11.263 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.253+0000 7fba9ddfb640 1 Processor -- start 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.253+0000 7fba9ddfb640 1 -- start start 2024-05-08T15:51:11.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.253+0000 7fba9ddfb640 10 monclient: init 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 10 monclient: _add_conns ranks=[2,3,7,8,0,5,1,4,6] 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 10 monclient(hunting): picked mon.noname-b con 0x7fba980a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 10 monclient(hunting): picked mon.noname-g con 0x7fba980a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 10 monclient(hunting): picked mon.noname-i con 0x7fba98137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba980a0f60 con 0x7fba980a41b0 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba98136020 con 0x7fba980a49a0 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba981417a0 con 0x7fba98137940 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba9ddfb640 10 monclient(hunting): authenticate will time out at 1079.369015s 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba977fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fba980a41b0 0x7fba98141090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35340/0 (socket says 172.21.15.7:35340) 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba977fe640 1 -- 172.21.15.7:0/3933225651 learned_addr learned my addr 172.21.15.7:0/3933225651 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3737837154 0 0) 0x7fba980a0f60 con 0x7fba980a41b0 2024-05-08T15:51:11.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): my global_id is 94961 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba84003520 con 0x7fba980a41b0 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4287381568 0 0) 0x7fba84003520 con 0x7fba980a41b0 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fba980a0f60 con 0x7fba980a41b0 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fba88001a90 con 0x7fba980a41b0 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): dump: 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.266 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:11.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.254+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3445720993 0 0) 0x7fba981417a0 con 0x7fba98137940 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): my global_id is 114118 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba84003520 con 0x7fba98137940 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2148336190 0 0) 0x7fba98136020 con 0x7fba980a49a0 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient(hunting): my global_id is 75828 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba981417a0 con 0x7fba980a49a0 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 179626069 0 0) 0x7fba980a0f60 con 0x7fba980a41b0 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.81:6792/0 conn(0x7fba98137940 legacy=0x7fba981d4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.81:6790/0 conn(0x7fba980a49a0 legacy=0x7fba981d14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 monclient: found mon.b 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba981362a0 con 0x7fba980a41b0 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 5 monclient: authenticate success, global_id 94961 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 10 monclient: _renew_subs 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fba98136530 con 0x7fba980a41b0 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 10 monclient: _renew_subs 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fba981cc590 con 0x7fba980a41b0 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fba88004a90 con 0x7fba980a41b0 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fba88004f80 con 0x7fba980a41b0 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba95ffb640 4 set_mon_vals no callback set 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba7ffff640 10 monclient: dump: 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.270 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.255+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fba780065c0 con 0x7fba78002860 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fba88010300 con 0x7fba980a41b0 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(194..194 src has 1..194) ==== 3898+0+0 (unknown 996519501 0 0) 0x7fba88046ee0 con 0x7fba980a41b0 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1654594741 0 0) 0x7fba780065c0 con 0x7fba78002860 2024-05-08T15:51:11.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 10 monclient(hunting): my global_id is 94787 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba980a0f60 con 0x7fba78002860 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba97fff640 10 monclient: get_auth_request con 0x7fba84039cf0 auth_method 0 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2922358292 0 0) 0x7fba980a0f60 con 0x7fba78002860 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fba780065c0 con 0x7fba78002860 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fba8c001210 con 0x7fba78002860 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 10 monclient: dump: 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.272 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3764906806 0 0) 0x7fba780065c0 con 0x7fba78002860 2024-05-08T15:51:11.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.256+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba7800a150 con 0x7fba78002860 2024-05-08T15:51:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.257+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fba8c001500 con 0x7fba78002860 2024-05-08T15:51:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.257+0000 7fba7ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.257+0000 7fba7ffff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.257+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.7:6793/0 conn(0x7fba78002860 legacy=0x7fba78004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.257+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.7:6793/0 conn(0x7fba78002860 legacy=0x7fba78004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba9ddfb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fba78009e20 con 0x7fba780090e0 2024-05-08T15:51:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982506842 0 0) 0x7fba78009e20 con 0x7fba780090e0 2024-05-08T15:51:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba7ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba7ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba7ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba7ffff640 10 monclient(hunting): my global_id is 94796 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.263+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba780065c0 con 0x7fba780090e0 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3213669930 0 0) 0x7fba780065c0 con 0x7fba780090e0 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fba78009e20 con 0x7fba780090e0 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fba8c004d10 con 0x7fba780090e0 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 10 monclient: dump: 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.277 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 845267043 0 0) 0x7fba78009e20 con 0x7fba780090e0 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fba7800c390 con 0x7fba780090e0 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 536144672) 0x7fba8c0062c0 con 0x7fba780090e0 2024-05-08T15:51:11.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.7:6793/0 conn(0x7fba780090e0 legacy=0x7fba78009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.264+0000 7fba7ffff640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.7:6793/0 conn(0x7fba780090e0 legacy=0x7fba78009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 121816, 2024-05-08T15:51:11.279 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:11.280 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:11.281 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:11.282 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:11.283 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.284 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.286 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.287 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.289 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:11.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.292 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.294 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.296 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.298 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:11.298 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:11.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.81:6812/847627118 conn(0x7fba84039cf0 legacy=0x7fba8403c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 10 monclient: shutdown 2024-05-08T15:51:11.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 >> v1:172.21.15.7:6790/0 conn(0x7fba980a41b0 legacy=0x7fba98141090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba97fff640 1 -- 172.21.15.7:0/3933225651 reap_dead start 2024-05-08T15:51:11.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba97fff640 1 -- 172.21.15.7:0/3933225651 reap_dead start 2024-05-08T15:51:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 shutdown_connections 2024-05-08T15:51:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 >> 172.21.15.7:0/3933225651 conn(0x7fba980a9fa0 msgr2=0x7fba980a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 shutdown_connections 2024-05-08T15:51:11.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.265+0000 7fba9ddfb640 1 -- 172.21.15.7:0/3933225651 wait complete. 2024-05-08T15:51:11.300 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:11.300 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:51:11.300 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:51:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.409+0000 7f6a67d4b640 10 monclient: get_monmap_and_config 2024-05-08T15:51:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.409+0000 7f6a67d4b640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.409+0000 7f6a67d4b640 10 monclient: monmap: 2024-05-08T15:51:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:11.410424+0000 2024-05-08T15:51:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:11.410424+0000 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.406 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 1 Processor -- start 2024-05-08T15:51:11.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 1 -- start start 2024-05-08T15:51:11.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient: init 2024-05-08T15:51:11.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient: _add_conns ranks=[5,7,4,0,1,8,3,6,2] 2024-05-08T15:51:11.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient(hunting): picked mon.noname-h con 0x7f6a60137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:11.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient(hunting): picked mon.noname-i con 0x7f6a600a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient(hunting): picked mon.noname-c con 0x7f6a600a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a600a2c70 con 0x7f6a600a41b0 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a600a2f50 con 0x7f6a60137940 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a600a1670 con 0x7f6a600a49a0 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a67d4b640 10 monclient(hunting): authenticate will time out at 1079.525495s 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a662e8640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f6a600a41b0 0x7f6a60140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55376/0 (socket says 172.21.15.7:55376) 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.410+0000 7f6a662e8640 1 -- 172.21.15.7:0/1172681394 learned_addr learned my addr 172.21.15.7:0/1172681394 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2389303503 0 0) 0x7f6a600a2c70 con 0x7f6a600a41b0 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): my global_id is 114160 2024-05-08T15:51:11.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a4c003520 con 0x7f6a600a41b0 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180961862 0 0) 0x7f6a600a2f50 con 0x7f6a60137940 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): my global_id is 5603 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a600a2c70 con 0x7f6a60137940 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3223576778 0 0) 0x7f6a600a1670 con 0x7f6a600a49a0 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient(hunting): my global_id is 114127 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a600a2f50 con 0x7f6a600a49a0 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2582197427 0 0) 0x7f6a4c003520 con 0x7f6a600a41b0 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 >> v1:172.21.15.81:6792/0 conn(0x7f6a600a49a0 legacy=0x7f6a6013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 >> v1:172.21.15.81:6791/0 conn(0x7f6a60137940 legacy=0x7f6a60137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 monclient: found mon.noname-c 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a600a39f0 con 0x7f6a600a41b0 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a58001b30 con 0x7f6a600a41b0 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: dump: 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.418 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a67d4b640 5 monclient: authenticate success, global_id 114160 2024-05-08T15:51:11.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a67d4b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6a58001e50 con 0x7f6a600a41b0 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 1 -- 172.21.15.7:0/1172681394 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a580020c0 con 0x7f6a600a41b0 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a67d4b640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a67d4b640 4 set_mon_vals no callback set 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.411+0000 7f6a64ae5640 10 monclient: dump: 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.420 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 10 monclient: shutdown 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 1 -- 172.21.15.7:0/1172681394 >> v1:172.21.15.7:6791/0 conn(0x7f6a600a41b0 legacy=0x7f6a60140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 1 -- 172.21.15.7:0/1172681394 shutdown_connections 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 1 -- 172.21.15.7:0/1172681394 >> 172.21.15.7:0/1172681394 conn(0x7f6a600a9fa0 msgr2=0x7f6a600aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 1 -- 172.21.15.7:0/1172681394 shutdown_connections 2024-05-08T15:51:11.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 1 -- 172.21.15.7:0/1172681394 wait complete. 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.412+0000 7f6a67d4b640 10 monclient: monmap: 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.422 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 1 Processor -- start 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 1 -- start start 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient: init 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient: _add_conns ranks=[8,2,4,1,7,3,0,6,5] 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient(hunting): picked mon.noname-e con 0x7f6a600a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient(hunting): picked mon.noname-b con 0x7f6a600a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient(hunting): picked mon.noname-c con 0x7f6a60137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a60139d20 con 0x7f6a600a49a0 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a601370e0 con 0x7f6a60137940 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a6013a000 con 0x7f6a600a41b0 2024-05-08T15:51:11.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a67d4b640 10 monclient(hunting): authenticate will time out at 1079.528405s 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a652e6640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f6a600a49a0 0x7f6a601356d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35352/0 (socket says 172.21.15.7:35352) 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a652e6640 1 -- 172.21.15.7:0/402202721 learned_addr learned my addr 172.21.15.7:0/402202721 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.413+0000 7f6a65ae7640 1 --1- 172.21.15.7:0/402202721 >> v1:172.21.15.7:6793/0 conn(0x7f6a600a41b0 0x7f6a60134fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50392/0 (socket says 172.21.15.7:50392) 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2911811110 0 0) 0x7f6a601370e0 con 0x7f6a60137940 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): my global_id is 114169 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a44003520 con 0x7f6a60137940 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840087331 0 0) 0x7f6a60139d20 con 0x7f6a600a49a0 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): my global_id is 94970 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a601370e0 con 0x7f6a600a49a0 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815677818 0 0) 0x7f6a6013a000 con 0x7f6a600a41b0 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): my global_id is 94805 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a60139d20 con 0x7f6a600a41b0 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3369576150 0 0) 0x7f6a44003520 con 0x7f6a60137940 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a6013a000 con 0x7f6a60137940 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a58001af0 con 0x7f6a60137940 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): dump: 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.425 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.426 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2423132937 0 0) 0x7f6a601370e0 con 0x7f6a600a49a0 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a44003520 con 0x7f6a600a49a0 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2552676702 0 0) 0x7f6a60139d20 con 0x7f6a600a41b0 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a601370e0 con 0x7f6a600a41b0 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a50001b10 con 0x7f6a600a49a0 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): dump: 2024-05-08T15:51:11.427 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.428 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a5c001850 con 0x7f6a600a41b0 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient(hunting): dump: 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.429 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2543396346 0 0) 0x7f6a6013a000 con 0x7f6a60137940 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.7:6793/0 conn(0x7f6a600a41b0 legacy=0x7f6a60134fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.7:6790/0 conn(0x7f6a600a49a0 legacy=0x7f6a601356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 monclient: found mon.c 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a600a0f60 con 0x7f6a60137940 2024-05-08T15:51:11.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a67d4b640 5 monclient: authenticate success, global_id 114169 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a67d4b640 10 monclient: _renew_subs 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a67d4b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6a60135de0 con 0x7f6a60137940 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a67d4b640 10 monclient: _renew_subs 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a67d4b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f6a6013a420 con 0x7f6a60137940 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6a580013a0 con 0x7f6a60137940 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a58002000 con 0x7f6a60137940 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:11.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a57fff640 4 set_mon_vals no callback set 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.414+0000 7f6a567fc640 10 monclient: dump: 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.432 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.415+0000 7f6a67d4b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.415+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6a480065c0 con 0x7f6a48002860 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.415+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6a580108a0 con 0x7f6a60137940 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.415+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(194..194 src has 1..194) ==== 3898+0+0 (unknown 996519501 0 0) 0x7f6a580470e0 con 0x7f6a60137940 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a652e6640 10 monclient: get_auth_request con 0x7f6a4403a5c0 auth_method 0 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1074094506 0 0) 0x7f6a480065c0 con 0x7f6a48002860 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient(hunting): my global_id is 5581 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a6013a000 con 0x7f6a48002860 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1207613496 0 0) 0x7f6a6013a000 con 0x7f6a48002860 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a480065c0 con 0x7f6a48002860 2024-05-08T15:51:11.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a5c003930 con 0x7f6a48002860 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient: dump: 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.434 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2985792846 0 0) 0x7f6a480065c0 con 0x7f6a48002860 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6a4800a150 con 0x7f6a48002860 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f6a5c005780 con 0x7f6a48002860 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.81:6789/0 conn(0x7f6a48002860 legacy=0x7f6a48004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.416+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.81:6789/0 conn(0x7f6a48002860 legacy=0x7f6a48004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a67d4b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f6a48009e40 con 0x7f6a48009100 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1488671135 0 0) 0x7f6a48009e40 con 0x7f6a48009100 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a567fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a567fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a567fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a567fc640 10 monclient(hunting): my global_id is 5590 2024-05-08T15:51:11.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.423+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a480065c0 con 0x7f6a48009100 2024-05-08T15:51:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1194746454 0 0) 0x7f6a480065c0 con 0x7f6a48009100 2024-05-08T15:51:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6a48009e40 con 0x7f6a48009100 2024-05-08T15:51:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6a5c004140 con 0x7f6a48009100 2024-05-08T15:51:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:51:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 10 monclient: dump: 2024-05-08T15:51:11.438 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.439 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 403145672 0 0) 0x7f6a48009e40 con 0x7f6a48009100 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6a4800c3d0 con 0x7f6a48009100 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7695 (unknown 0 0 4073401323) 0x7f6a5c004140 con 0x7f6a48009100 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.81:6789/0 conn(0x7f6a48009100 legacy=0x7f6a48009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.424+0000 7f6a567fc640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.81:6789/0 conn(0x7f6a48009100 legacy=0x7f6a48009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:11.440 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 263740, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.441 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.442 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.443 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:11.444 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.445 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.446 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:11.447 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:11.448 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.449 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.450 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.451 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.452 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:11.453 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.454 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.455 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.456 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.457 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.458 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:11.459 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.81:6812/847627118 conn(0x7f6a4403a5c0 legacy=0x7f6a4403ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 10 monclient: shutdown 2024-05-08T15:51:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 >> v1:172.21.15.7:6791/0 conn(0x7f6a60137940 legacy=0x7f6a601d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a662e8640 1 -- 172.21.15.7:0/402202721 reap_dead start 2024-05-08T15:51:11.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a662e8640 1 -- 172.21.15.7:0/402202721 reap_dead start 2024-05-08T15:51:11.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 shutdown_connections 2024-05-08T15:51:11.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 >> 172.21.15.7:0/402202721 conn(0x7f6a600a9fa0 msgr2=0x7f6a600a88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 shutdown_connections 2024-05-08T15:51:11.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.425+0000 7f6a67d4b640 1 -- 172.21.15.7:0/402202721 wait complete. 2024-05-08T15:51:11.461 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:11.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:51:11.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:51:11.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.569+0000 7fa9ade55640 10 monclient: get_monmap_and_config 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.569+0000 7fa9ade55640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.569+0000 7fa9ade55640 10 monclient: monmap: 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:11.570895+0000 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:11.570895+0000 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.566 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.567 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.567 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.570+0000 7fa9ade55640 1 Processor -- start 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.570+0000 7fa9ade55640 1 -- start start 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.570+0000 7fa9ade55640 10 monclient: init 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 10 monclient: _add_conns ranks=[4,1,3,2,0,8,7,5,6] 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 10 monclient(hunting): picked mon.noname-c con 0x7fa9a8137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 10 monclient(hunting): picked mon.noname-f con 0x7fa9a80a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 10 monclient(hunting): picked mon.noname-g con 0x7fa9a80a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a80a2c70 con 0x7fa9a80a49a0 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a80a2f50 con 0x7fa9a80a41b0 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a80a1670 con 0x7fa9a8137940 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 10 monclient(hunting): authenticate will time out at 1079.685888s 2024-05-08T15:51:11.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a77fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fa9a8137940 0x7fa9a8137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55402/0 (socket says 172.21.15.7:55402) 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a77fe640 1 -- 172.21.15.7:0/2563311704 learned_addr learned my addr 172.21.15.7:0/2563311704 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062718027 0 0) 0x7fa9a80a1670 con 0x7fa9a8137940 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient(hunting): my global_id is 114178 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa994003520 con 0x7fa9a8137940 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 328395892 0 0) 0x7fa994003520 con 0x7fa9a8137940 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 >> v1:172.21.15.81:6790/0 conn(0x7fa9a80a41b0 legacy=0x7fa9a8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 >> v1:172.21.15.81:6789/0 conn(0x7fa9a80a49a0 legacy=0x7fa9a813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 monclient: found mon.noname-c 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a80a39f0 con 0x7fa9a8137940 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa9980017b0 con 0x7fa9a8137940 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9a67fc640 10 monclient: dump: 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.577 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 5 monclient: authenticate success, global_id 114178 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.571+0000 7fa9ade55640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa998001a90 con 0x7fa9a8137940 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9a67fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9a67fc640 1 -- 172.21.15.7:0/2563311704 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa998004be0 con 0x7fa9a8137940 2024-05-08T15:51:11.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 4 set_mon_vals no callback set 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9a67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9a67fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9a67fc640 10 monclient: dump: 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.579 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 10 monclient: shutdown 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 1 -- 172.21.15.7:0/2563311704 >> v1:172.21.15.7:6791/0 conn(0x7fa9a8137940 legacy=0x7fa9a8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 1 -- 172.21.15.7:0/2563311704 shutdown_connections 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 1 -- 172.21.15.7:0/2563311704 >> 172.21.15.7:0/2563311704 conn(0x7fa9a80a9fa0 msgr2=0x7fa9a80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 1 -- 172.21.15.7:0/2563311704 shutdown_connections 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.572+0000 7fa9ade55640 1 -- 172.21.15.7:0/2563311704 wait complete. 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.573+0000 7fa9ade55640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.573+0000 7fa9ade55640 10 monclient: monmap: 2024-05-08T15:51:11.580 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.573+0000 7fa9ade55640 1 Processor -- start 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.573+0000 7fa9ade55640 1 -- start start 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.573+0000 7fa9ade55640 10 monclient: init 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 10 monclient: _add_conns ranks=[0,8,4,6,2,5,1,3,7] 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 10 monclient(hunting): picked mon.noname-a con 0x7fa9a80a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 10 monclient(hunting): picked mon.noname-e con 0x7fa9a80a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 10 monclient(hunting): picked mon.noname-c con 0x7fa9a8137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a80a0f60 con 0x7fa9a80a41b0 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a81369a0 con 0x7fa9a8137940 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9a81394b0 con 0x7fa9a80a49a0 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9ade55640 10 monclient(hunting): authenticate will time out at 1079.688763s 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9a7fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fa9a8137940 0x7fa9a81d4eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55414/0 (socket says 172.21.15.7:55414) 2024-05-08T15:51:11.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa9a7fff640 1 -- 172.21.15.7:0/4123073642 learned_addr learned my addr 172.21.15.7:0/4123073642 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1057816403 0 0) 0x7fa9a81369a0 con 0x7fa9a8137940 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): my global_id is 114187 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa988003520 con 0x7fa9a8137940 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2131733348 0 0) 0x7fa9a81394b0 con 0x7fa9a80a49a0 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): my global_id is 94814 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9a81369a0 con 0x7fa9a80a49a0 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1105772295 0 0) 0x7fa9a80a0f60 con 0x7fa9a80a41b0 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): my global_id is 94851 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9a81394b0 con 0x7fa9a80a41b0 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1329274485 0 0) 0x7fa988003520 con 0x7fa9a8137940 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa9a80a0f60 con 0x7fa9a8137940 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa9a0001e20 con 0x7fa9a8137940 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): dump: 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3301133585 0 0) 0x7fa9a81394b0 con 0x7fa9a80a41b0 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa988003520 con 0x7fa9a80a41b0 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2042237919 0 0) 0x7fa9a81369a0 con 0x7fa9a80a49a0 2024-05-08T15:51:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa9a81394b0 con 0x7fa9a80a49a0 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa9980016b0 con 0x7fa9a80a41b0 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): dump: 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa990001990 con 0x7fa9a80a49a0 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.574+0000 7fa99ffff640 10 monclient(hunting): dump: 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1944773353 0 0) 0x7fa9a80a0f60 con 0x7fa9a8137940 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.7:6793/0 conn(0x7fa9a80a49a0 legacy=0x7fa9a81d1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.7:6789/0 conn(0x7fa9a80a41b0 legacy=0x7fa9a81412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 1 monclient: found mon.c 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a81419f0 con 0x7fa9a8137940 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9ade55640 5 monclient: authenticate success, global_id 114187 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9ade55640 10 monclient: _renew_subs 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9ade55640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a8138a00 con 0x7fa9a8137940 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9ade55640 10 monclient: _renew_subs 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9ade55640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fa9a81376e0 con 0x7fa9a8137940 2024-05-08T15:51:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa9a0003c40 con 0x7fa9a8137940 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa9a0004130 con 0x7fa9a8137940 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa9a5ffb640 4 set_mon_vals no callback set 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.575+0000 7fa99ffff640 10 monclient: dump: 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa9ade55640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa98c0065c0 con 0x7fa98c002860 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa9a00042d0 con 0x7fa9a8137940 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(194..194 src has 1..194) ==== 3898+0+0 (unknown 996519501 0 0) 0x7fa9a0047250 con 0x7fa9a8137940 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2818375623 0 0) 0x7fa98c0065c0 con 0x7fa98c002860 2024-05-08T15:51:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 10 monclient(hunting): my global_id is 75846 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9a80a0f60 con 0x7fa98c002860 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa9a6ffd640 10 monclient: get_auth_request con 0x7fa98803a490 auth_method 0 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 899907072 0 0) 0x7fa9a80a0f60 con 0x7fa98c002860 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa98c0065c0 con 0x7fa98c002860 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa998004b40 con 0x7fa98c002860 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.576+0000 7fa99ffff640 10 monclient: dump: 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.592 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.577+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2021063822 0 0) 0x7fa98c0065c0 con 0x7fa98c002860 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.577+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa98c00a150 con 0x7fa98c002860 2024-05-08T15:51:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.577+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fa998007480 con 0x7fa98c002860 2024-05-08T15:51:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.577+0000 7fa99ffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.577+0000 7fa99ffff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.577+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.81:6790/0 conn(0x7fa98c002860 legacy=0x7fa98c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.577+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.81:6790/0 conn(0x7fa98c002860 legacy=0x7fa98c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa9ade55640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa98c009e40 con 0x7fa98c009100 2024-05-08T15:51:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656828308 0 0) 0x7fa98c009e40 con 0x7fa98c009100 2024-05-08T15:51:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 10 monclient(hunting): my global_id is 75855 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa98c0065c0 con 0x7fa98c009100 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3867413699 0 0) 0x7fa98c0065c0 con 0x7fa98c009100 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa98c009e40 con 0x7fa98c009100 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa998005420 con 0x7fa98c009100 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 10 monclient: dump: 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2084553385 0 0) 0x7fa98c009e40 con 0x7fa98c009100 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.584+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa98c00c3d0 con 0x7fa98c009100 2024-05-08T15:51:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1866831338) 0x7fa998006870 con 0x7fa98c009100 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa99ffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa99ffff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.81:6790/0 conn(0x7fa98c009100 legacy=0x7fa98c009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa99ffff640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.81:6790/0 conn(0x7fa98c009100 legacy=0x7fa98c009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:11.598 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 263875, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:11.599 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:11.600 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:11.601 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:11.602 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.603 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.604 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.605 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.606 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.607 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:11.608 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.609 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.610 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.611 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:11.612 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.613 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.614 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.615 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:11.616 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:11.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.81:6812/847627118 conn(0x7fa98803a490 legacy=0x7fa98803c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9ade55640 10 monclient: shutdown 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9ade55640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 >> v1:172.21.15.7:6791/0 conn(0x7fa9a8137940 legacy=0x7fa9a81d4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9a7fff640 1 -- 172.21.15.7:0/4123073642 reap_dead start 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9a7fff640 1 -- 172.21.15.7:0/4123073642 reap_dead start 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 shutdown_connections 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 >> 172.21.15.7:0/4123073642 conn(0x7fa9a80a9fa0 msgr2=0x7fa9a80a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.585+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 shutdown_connections 2024-05-08T15:51:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.586+0000 7fa9ade55640 1 -- 172.21.15.7:0/4123073642 wait complete. 2024-05-08T15:51:11.619 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:11.619 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:51:11.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.727+0000 7f5096206640 10 monclient: get_monmap_and_config 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.727+0000 7f5096206640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.727+0000 7f5096206640 10 monclient: monmap: 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:11.728181+0000 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:11.728181+0000 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.723 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.724 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.727+0000 7f5096206640 1 Processor -- start 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.727+0000 7f5096206640 1 -- start start 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.727+0000 7f5096206640 10 monclient: init 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 10 monclient: _add_conns ranks=[4,2,7,8,6,5,3,1,0] 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 10 monclient(hunting): picked mon.noname-c con 0x7f5090137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 10 monclient(hunting): picked mon.noname-b con 0x7f50900a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 10 monclient(hunting): picked mon.noname-i con 0x7f50900a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50900a2c70 con 0x7f50900a49a0 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50900a2f50 con 0x7f5090137940 2024-05-08T15:51:11.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50900a1670 con 0x7f50900a41b0 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5096206640 10 monclient(hunting): authenticate will time out at 1079.843126s 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5086ffd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f50900a49a0 0x7f509013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35358/0 (socket says 172.21.15.7:35358) 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f5086ffd640 1 -- 172.21.15.7:0/1635659928 learned_addr learned my addr 172.21.15.7:0/1635659928 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1443132450 0 0) 0x7f50900a2c70 con 0x7f50900a49a0 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): my global_id is 94979 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f507c003520 con 0x7f50900a49a0 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065270941 0 0) 0x7f50900a2f50 con 0x7f5090137940 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 10 monclient(hunting): my global_id is 114196 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.728+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50900a2c70 con 0x7f5090137940 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3800472447 0 0) 0x7f507c003520 con 0x7f50900a49a0 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 >> v1:172.21.15.81:6792/0 conn(0x7f50900a41b0 legacy=0x7f5090140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 >> v1:172.21.15.7:6791/0 conn(0x7f5090137940 legacy=0x7f5090137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 monclient: found mon.noname-b 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50900a39f0 con 0x7f50900a49a0 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 5 monclient: authenticate success, global_id 94979 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f50780017b0 con 0x7f50900a49a0 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: dump: 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.736 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5078001a90 con 0x7f50900a49a0 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 1 -- 172.21.15.7:0/1635659928 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5078004be0 con 0x7f50900a49a0 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f508effd640 10 monclient: dump: 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.737 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 4 set_mon_vals no callback set 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 10 monclient: shutdown 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 1 -- 172.21.15.7:0/1635659928 >> v1:172.21.15.7:6790/0 conn(0x7f50900a49a0 legacy=0x7f509013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 1 -- 172.21.15.7:0/1635659928 shutdown_connections 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 1 -- 172.21.15.7:0/1635659928 >> 172.21.15.7:0/1635659928 conn(0x7f50900a9fa0 msgr2=0x7f50900aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 1 -- 172.21.15.7:0/1635659928 shutdown_connections 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.729+0000 7f5096206640 1 -- 172.21.15.7:0/1635659928 wait complete. 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.730+0000 7f5096206640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.730+0000 7f5096206640 10 monclient: monmap: 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:11.739 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.740 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.730+0000 7f5096206640 1 Processor -- start 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 1 -- start start 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient: init 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient: _add_conns ranks=[7,2,8,3,5,4,0,6,1] 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient(hunting): picked mon.noname-i con 0x7f50900a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient(hunting): picked mon.noname-b con 0x7f50900a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient(hunting): picked mon.noname-e con 0x7f5090137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:11.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50900a1f30 con 0x7f50900a49a0 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50900a0f60 con 0x7f50900a41b0 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5086ffd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f50900a49a0 0x7f50901d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35362/0 (socket says 172.21.15.7:35362) 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5086ffd640 1 -- 172.21.15.7:0/3991474698 learned_addr learned my addr 172.21.15.7:0/3991474698 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50901c9800 con 0x7f5090137940 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f5096206640 10 monclient(hunting): authenticate will time out at 1079.846227s 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734716916 0 0) 0x7f50900a1f30 con 0x7f50900a49a0 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): my global_id is 94988 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5070003520 con 0x7f50900a49a0 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312494108 0 0) 0x7f50901c9800 con 0x7f5090137940 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 10 monclient(hunting): my global_id is 94823 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.731+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50900a1f30 con 0x7f5090137940 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2788665171 0 0) 0x7f50900a0f60 con 0x7f50900a41b0 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): my global_id is 114145 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50901c9800 con 0x7f50900a41b0 2024-05-08T15:51:11.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3943913098 0 0) 0x7f5070003520 con 0x7f50900a49a0 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f50900a0f60 con 0x7f50900a49a0 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5078004b20 con 0x7f50900a49a0 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): dump: 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.744 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2321800360 0 0) 0x7f50900a1f30 con 0x7f5090137940 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5070003520 con 0x7f5090137940 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5088001c90 con 0x7f5090137940 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient(hunting): dump: 2024-05-08T15:51:11.745 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.746 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2898834637 0 0) 0x7f50900a0f60 con 0x7f50900a49a0 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.7:6793/0 conn(0x7f5090137940 legacy=0x7f50901d4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.81:6792/0 conn(0x7f50900a41b0 legacy=0x7f5090141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 monclient: found mon.b 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5090135790 con 0x7f50900a49a0 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f5096206640 5 monclient: authenticate success, global_id 94988 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f5096206640 10 monclient: _renew_subs 2024-05-08T15:51:11.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f5096206640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5090138ba0 con 0x7f50900a49a0 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f5096206640 10 monclient: _renew_subs 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f5096206640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5090141850 con 0x7f50900a49a0 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5078001a90 con 0x7f50900a49a0 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5078001c40 con 0x7f50900a49a0 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508e7fc640 4 set_mon_vals no callback set 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.732+0000 7f508cff9640 10 monclient: dump: 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.748 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f5096206640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f50740065c0 con 0x7f5074002860 2024-05-08T15:51:11.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f5078010700 con 0x7f50900a49a0 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(194..194 src has 1..194) ==== 3898+0+0 (unknown 996519501 0 0) 0x7f5078046eb0 con 0x7f50900a49a0 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508ffff640 10 monclient: get_auth_request con 0x7f507003a080 auth_method 0 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2609541760 0 0) 0x7f50740065c0 con 0x7f5074002860 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 10 monclient(hunting): my global_id is 5612 2024-05-08T15:51:11.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.733+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50900a0f60 con 0x7f5074002860 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2694229300 0 0) 0x7f50900a0f60 con 0x7f5074002860 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f50740065c0 con 0x7f5074002860 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5080003930 con 0x7f5074002860 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 10 monclient: dump: 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.751 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 815632060 0 0) 0x7f50740065c0 con 0x7f5074002860 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f507400a150 con 0x7f5074002860 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5080005b50 con 0x7f5074002860 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.81:6791/0 conn(0x7f5074002860 legacy=0x7f5074004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.734+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.81:6791/0 conn(0x7f5074002860 legacy=0x7f5074004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f5096206640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5074009e40 con 0x7f5074009100 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4233846143 0 0) 0x7f5074009e40 con 0x7f5074009100 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f508cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f508cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f508cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f508cff9640 10 monclient(hunting): my global_id is 5621 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.741+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50740065c0 con 0x7f5074009100 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3546772815 0 0) 0x7f50740065c0 con 0x7f5074009100 2024-05-08T15:51:11.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5074009e40 con 0x7f5074009100 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5080005340 con 0x7f5074009100 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 10 monclient: dump: 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.755 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3605254657 0 0) 0x7f5074009e40 con 0x7f5074009100 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f507400c3d0 con 0x7f5074009100 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 296274304) 0x7f50800055e0 con 0x7f5074009100 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.81:6791/0 conn(0x7f5074009100 legacy=0x7f5074009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.742+0000 7f508cff9640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.81:6791/0 conn(0x7f5074009100 legacy=0x7f5074009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 264048, 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:11.757 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.758 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.759 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:11.760 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:11.761 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.762 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.763 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.766 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.767 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.771 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.772 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.773 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.774 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:11.775 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:11.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.81:6812/847627118 conn(0x7f507003a080 legacy=0x7f507003c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 10 monclient: shutdown 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 >> v1:172.21.15.7:6790/0 conn(0x7f50900a49a0 legacy=0x7f50901d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f508ffff640 1 -- 172.21.15.7:0/3991474698 reap_dead start 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f508ffff640 1 -- 172.21.15.7:0/3991474698 reap_dead start 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 shutdown_connections 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 >> 172.21.15.7:0/3991474698 conn(0x7f50900a9fa0 msgr2=0x7f5090130760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 shutdown_connections 2024-05-08T15:51:11.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.743+0000 7f5096206640 1 -- 172.21.15.7:0/3991474698 wait complete. 2024-05-08T15:51:11.777 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:11.778 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:51:11.778 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:51:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.886+0000 7fc6a4550640 10 monclient: get_monmap_and_config 2024-05-08T15:51:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.886+0000 7fc6a4550640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.886+0000 7fc6a4550640 10 monclient: monmap: 2024-05-08T15:51:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.882 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:11.887057+0000 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:11.887057+0000 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.886+0000 7fc6a4550640 1 Processor -- start 2024-05-08T15:51:11.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.886+0000 7fc6a4550640 1 -- start start 2024-05-08T15:51:11.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.886+0000 7fc6a4550640 10 monclient: init 2024-05-08T15:51:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 10 monclient: _add_conns ranks=[6,2,8,3,4,0,7,5,1] 2024-05-08T15:51:11.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 10 monclient(hunting): picked mon.noname-d con 0x7fc69c0a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 10 monclient(hunting): picked mon.noname-b con 0x7fc69c0a5780 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 10 monclient(hunting): picked mon.noname-e con 0x7fc69c13e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c0a1660 con 0x7fc69c0a5780 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c0a28f0 con 0x7fc69c0a53a0 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c0a2bd0 con 0x7fc69c13e310 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a4550640 10 monclient(hunting): authenticate will time out at 1080.001987s 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a22ec640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fc69c0a53a0 0x7fc69c0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:46884/0 (socket says 172.21.15.7:46884) 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a22ec640 1 -- 172.21.15.7:0/1827690819 learned_addr learned my addr 172.21.15.7:0/1827690819 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3611583324 0 0) 0x7fc69c0a2bd0 con 0x7fc69c13e310 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): my global_id is 94832 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc688003520 con 0x7fc69c13e310 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1915063163 0 0) 0x7fc69c0a28f0 con 0x7fc69c0a53a0 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): my global_id is 75696 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc69c0a2bd0 con 0x7fc69c0a53a0 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1138217332 0 0) 0x7fc69c0a1660 con 0x7fc69c0a5780 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 10 monclient(hunting): my global_id is 94997 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.887+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc69c0a28f0 con 0x7fc69c0a5780 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 231097645 0 0) 0x7fc688003520 con 0x7fc69c13e310 2024-05-08T15:51:11.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 >> v1:172.21.15.7:6792/0 conn(0x7fc69c0a53a0 legacy=0x7fc69c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 >> v1:172.21.15.7:6790/0 conn(0x7fc69c0a5780 legacy=0x7fc69c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 monclient: found mon.noname-e 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c0a6730 con 0x7fc69c13e310 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 5 monclient: authenticate success, global_id 94832 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc6940017b0 con 0x7fc69c13e310 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: dump: 2024-05-08T15:51:11.894 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc694001a50 con 0x7fc69c13e310 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 1 -- 172.21.15.7:0/1827690819 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc694003950 con 0x7fc69c13e310 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 4 set_mon_vals no callback set 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a12ea640 10 monclient: dump: 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.897 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 10 monclient: shutdown 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 1 -- 172.21.15.7:0/1827690819 >> v1:172.21.15.7:6793/0 conn(0x7fc69c13e310 legacy=0x7fc69c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 1 -- 172.21.15.7:0/1827690819 shutdown_connections 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 1 -- 172.21.15.7:0/1827690819 >> 172.21.15.7:0/1827690819 conn(0x7fc69c0a9fa0 msgr2=0x7fc69c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 1 -- 172.21.15.7:0/1827690819 shutdown_connections 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.888+0000 7fc6a4550640 1 -- 172.21.15.7:0/1827690819 wait complete. 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.889+0000 7fc6a4550640 10 monclient: build_initial_monmap 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.889+0000 7fc6a4550640 10 monclient: monmap: 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:11.898 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.889+0000 7fc6a4550640 1 Processor -- start 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.889+0000 7fc6a4550640 1 -- start start 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.889+0000 7fc6a4550640 10 monclient: init 2024-05-08T15:51:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 10 monclient: _add_conns ranks=[5,4,3,2,1,0,6,7,8] 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 10 monclient(hunting): picked mon.noname-h con 0x7fc69c0a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 10 monclient(hunting): picked mon.noname-c con 0x7fc69c0a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 10 monclient(hunting): picked mon.noname-g con 0x7fc69c13e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c137550 con 0x7fc69c13e310 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c1ccf80 con 0x7fc69c0a5780 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc69c138780 con 0x7fc69c0a53a0 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a4550640 10 monclient(hunting): authenticate will time out at 1080.004920s 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a1aeb640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fc69c0a5780 0x7fc69c1cc870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55432/0 (socket says 172.21.15.7:55432) 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc6a1aeb640 1 -- 172.21.15.7:0/295983383 learned_addr learned my addr 172.21.15.7:0/295983383 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2981547433 0 0) 0x7fc69c1ccf80 con 0x7fc69c0a5780 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): my global_id is 114205 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc680003520 con 0x7fc69c0a5780 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1563858455 0 0) 0x7fc680003520 con 0x7fc69c0a5780 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc69c1ccf80 con 0x7fc69c0a5780 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc68c0016f0 con 0x7fc69c0a5780 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): dump: 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4073507763 0 0) 0x7fc69c138780 con 0x7fc69c0a53a0 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): my global_id is 5630 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc680003520 con 0x7fc69c0a53a0 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 410067732 0 0) 0x7fc69c137550 con 0x7fc69c13e310 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 10 monclient(hunting): my global_id is 75864 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.890+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc69c138780 con 0x7fc69c13e310 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2500279148 0 0) 0x7fc69c1ccf80 con 0x7fc69c0a5780 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.81:6791/0 conn(0x7fc69c0a53a0 legacy=0x7fc69c1cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.81:6790/0 conn(0x7fc69c13e310 legacy=0x7fc69c1d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 monclient: found mon.c 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc69c135540 con 0x7fc69c0a5780 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: _finish_auth 0 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 5 monclient: authenticate success, global_id 114205 2024-05-08T15:51:11.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 10 monclient: _renew_subs 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc69c136f90 con 0x7fc69c0a5780 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 10 monclient: _renew_subs 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fc69c133900 con 0x7fc69c0a5780 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc68c003920 con 0x7fc69c0a5780 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc68c003e90 con 0x7fc69c0a5780 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a0ae9640 4 set_mon_vals no callback set 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 10 monclient: dump: 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.905 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.906 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc6840065c0 con 0x7fc684002860 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.891+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fc68c004030 con 0x7fc69c0a5780 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(194..194 src has 1..194) ==== 3898+0+0 (unknown 996519501 0 0) 0x7fc68c046f00 con 0x7fc69c0a5780 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748119631 0 0) 0x7fc6840065c0 con 0x7fc684002860 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 10 monclient(hunting): my global_id is 114154 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc69c1ccf80 con 0x7fc684002860 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc6a2aed640 10 monclient: get_auth_request con 0x7fc680039ca0 auth_method 0 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2199307707 0 0) 0x7fc69c1ccf80 con 0x7fc684002860 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc6840065c0 con 0x7fc684002860 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc6980017a0 con 0x7fc684002860 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 10 monclient: dump: 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.908 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2337094037 0 0) 0x7fc6840065c0 con 0x7fc684002860 2024-05-08T15:51:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.892+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc68400a150 con 0x7fc684002860 2024-05-08T15:51:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.893+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fc698001a80 con 0x7fc684002860 2024-05-08T15:51:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.893+0000 7fc692ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.893+0000 7fc692ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.893+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.81:6792/0 conn(0x7fc684002860 legacy=0x7fc684004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.893+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.81:6792/0 conn(0x7fc684002860 legacy=0x7fc684004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.899+0000 7fc6a4550640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.899+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc684009e40 con 0x7fc684009100 2024-05-08T15:51:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 658816363 0 0) 0x7fc684009e40 con 0x7fc684009100 2024-05-08T15:51:11.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 10 monclient(hunting): my global_id is 114163 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6840065c0 con 0x7fc684009100 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2210656133 0 0) 0x7fc6840065c0 con 0x7fc684009100 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc684009e40 con 0x7fc684009100 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc698005160 con 0x7fc684009100 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 10 monclient: dump: 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:11.912 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3748875236 0 0) 0x7fc684009e40 con 0x7fc684009100 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.900+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc68400c3d0 con 0x7fc684009100 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 1604393441) 0x7fc6980074d0 con 0x7fc684009100 2024-05-08T15:51:11.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc692ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc692ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.81:6792/0 conn(0x7fc684009100 legacy=0x7fc684009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc692ffd640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.81:6792/0 conn(0x7fc684009100 legacy=0x7fc684009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 6764, 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:11.914 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.915 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:11.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:11.917 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:11.918 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:11.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:11.920 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.922 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.925 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.926 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:11.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:11.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:11.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:11.932 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:11.932 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.81:6812/847627118 conn(0x7fc680039ca0 legacy=0x7fc68003c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 10 monclient: shutdown 2024-05-08T15:51:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 >> v1:172.21.15.7:6791/0 conn(0x7fc69c0a5780 legacy=0x7fc69c1cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a2aed640 1 -- 172.21.15.7:0/295983383 reap_dead start 2024-05-08T15:51:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a2aed640 1 -- 172.21.15.7:0/295983383 reap_dead start 2024-05-08T15:51:11.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 shutdown_connections 2024-05-08T15:51:11.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 >> 172.21.15.7:0/295983383 conn(0x7fc69c0a9fa0 msgr2=0x7fc69c140b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:11.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 shutdown_connections 2024-05-08T15:51:11.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:11.901+0000 7fc6a4550640 1 -- 172.21.15.7:0/295983383 wait complete. 2024-05-08T15:51:11.934 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:11.934 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T15:51:11.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T15:51:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.041+0000 7fd425159640 10 monclient: get_monmap_and_config 2024-05-08T15:51:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.041+0000 7fd425159640 10 monclient: build_initial_monmap 2024-05-08T15:51:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.041+0000 7fd425159640 10 monclient: monmap: 2024-05-08T15:51:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:12.042405+0000 2024-05-08T15:51:12.037 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:12.042405+0000 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:12.038 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 1 Processor -- start 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 1 -- start start 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient: init 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient: _add_conns ranks=[0,5,8,6,4,3,1,7,2] 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient(hunting): picked mon.noname-a con 0x7fd420137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient(hunting): picked mon.noname-h con 0x7fd4200a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient(hunting): picked mon.noname-e con 0x7fd4200a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4200a2c70 con 0x7fd420137940 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4200a2f50 con 0x7fd4200a49a0 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4200a1670 con 0x7fd4200a41b0 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:12.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd425159640 10 monclient(hunting): authenticate will time out at 1080.157194s 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd41ed9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd420137940 0x7fd420137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49746/0 (socket says 172.21.15.7:49746) 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd41f59e640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fd4200a41b0 0x7fd420140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50426/0 (socket says 172.21.15.7:50426) 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd41ed9d640 1 -- 172.21.15.7:0/82938153 learned_addr learned my addr 172.21.15.7:0/82938153 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.042+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623861128 0 0) 0x7fd4200a2c70 con 0x7fd420137940 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): my global_id is 94860 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd408003520 con 0x7fd420137940 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4003926242 0 0) 0x7fd4200a1670 con 0x7fd4200a41b0 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): my global_id is 94841 2024-05-08T15:51:12.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4200a2c70 con 0x7fd4200a41b0 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 573676408 0 0) 0x7fd4200a2f50 con 0x7fd4200a49a0 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient(hunting): my global_id is 5639 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4200a1670 con 0x7fd4200a49a0 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 246126663 0 0) 0x7fd4200a2c70 con 0x7fd4200a41b0 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 >> v1:172.21.15.81:6791/0 conn(0x7fd4200a49a0 legacy=0x7fd42013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 >> v1:172.21.15.7:6789/0 conn(0x7fd420137940 legacy=0x7fd420137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: _finish_hunting 0 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 monclient: found mon.noname-e 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4200a39f0 con 0x7fd4200a41b0 2024-05-08T15:51:12.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: _finish_auth 0 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: _check_auth_tickets 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd418001a90 con 0x7fd4200a41b0 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 5 monclient: authenticate success, global_id 94841 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: dump: 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd418001d70 con 0x7fd4200a41b0 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 1 -- 172.21.15.7:0/82938153 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd418004ea0 con 0x7fd4200a41b0 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd41dd9b640 10 monclient: dump: 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:12.051 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 4 set_mon_vals no callback set 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 10 monclient: shutdown 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 1 -- 172.21.15.7:0/82938153 >> v1:172.21.15.7:6793/0 conn(0x7fd4200a41b0 legacy=0x7fd420140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 1 -- 172.21.15.7:0/82938153 shutdown_connections 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.043+0000 7fd425159640 1 -- 172.21.15.7:0/82938153 >> 172.21.15.7:0/82938153 conn(0x7fd4200a9fa0 msgr2=0x7fd4200aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.044+0000 7fd425159640 1 -- 172.21.15.7:0/82938153 shutdown_connections 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.044+0000 7fd425159640 1 -- 172.21.15.7:0/82938153 wait complete. 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.044+0000 7fd425159640 10 monclient: build_initial_monmap 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.044+0000 7fd425159640 10 monclient: monmap: 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:12.052 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 1 Processor -- start 2024-05-08T15:51:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 1 -- start start 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient: init 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient: _add_conns ranks=[8,3,6,0,1,2,7,5,4] 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient(hunting): picked mon.noname-e con 0x7fd4200a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient(hunting): picked mon.noname-g con 0x7fd4200a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient(hunting): picked mon.noname-d con 0x7fd420137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4200a1f30 con 0x7fd4200a49a0 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4201cda40 con 0x7fd420137940 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd420136200 con 0x7fd4200a41b0 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd425159640 10 monclient(hunting): authenticate will time out at 1080.160350s 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd41ed9d640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fd4200a41b0 0x7fd4201ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50434/0 (socket says 172.21.15.7:50434) 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.045+0000 7fd41ed9d640 1 -- 172.21.15.7:0/614597006 learned_addr learned my addr 172.21.15.7:0/614597006 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840176438 0 0) 0x7fd4201cda40 con 0x7fd420137940 2024-05-08T15:51:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): my global_id is 75705 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd40c003520 con 0x7fd420137940 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173773614 0 0) 0x7fd420136200 con 0x7fd4200a41b0 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): my global_id is 94850 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4201cda40 con 0x7fd4200a41b0 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838130158 0 0) 0x7fd4200a1670 con 0x7fd4200a49a0 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:12.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): my global_id is 75873 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd420136200 con 0x7fd4200a49a0 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2528595794 0 0) 0x7fd4201cda40 con 0x7fd4200a41b0 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd4200a1670 con 0x7fd4200a41b0 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd410001940 con 0x7fd4200a41b0 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): dump: 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:12.056 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3788195596 0 0) 0x7fd40c003520 con 0x7fd420137940 2024-05-08T15:51:12.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd4201cda40 con 0x7fd420137940 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd418001710 con 0x7fd420137940 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): dump: 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:12.058 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4104653485 0 0) 0x7fd420136200 con 0x7fd4200a49a0 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd40c003520 con 0x7fd4200a49a0 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd414001990 con 0x7fd4200a49a0 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient(hunting): dump: 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.059 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2131946806 0 0) 0x7fd4200a1670 con 0x7fd4200a41b0 2024-05-08T15:51:12.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 >> v1:172.21.15.7:6792/0 conn(0x7fd420137940 legacy=0x7fd4201d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 >> v1:172.21.15.81:6790/0 conn(0x7fd4200a49a0 legacy=0x7fd4201cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 monclient: found mon.e 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd420133a80 con 0x7fd4200a41b0 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd425159640 5 monclient: authenticate success, global_id 94850 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd425159640 10 monclient: _renew_subs 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd425159640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd4201354b0 con 0x7fd4200a41b0 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd425159640 10 monclient: _renew_subs 2024-05-08T15:51:12.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd425159640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fd420134140 con 0x7fd4200a41b0 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd410001ca0 con 0x7fd4200a41b0 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd410004dd0 con 0x7fd4200a41b0 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd41d59a640 4 set_mon_vals no callback set 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.046+0000 7fd4077fe640 10 monclient: dump: 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:12.062 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.047+0000 7fd425159640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.047+0000 7fd425159640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:12.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.047+0000 7fd425159640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.047+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd410010150 con 0x7fd4200a41b0 2024-05-08T15:51:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.047+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd400005080 con 0x7fd4200a41b0 2024-05-08T15:51:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.047+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(194..194 src has 1..194) ==== 3898+0+0 (unknown 996519501 0 0) 0x7fd410046d70 con 0x7fd4200a41b0 2024-05-08T15:51:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.048+0000 7fd41e59c640 10 monclient: get_auth_request con 0x7fd40c03a520 auth_method 0 2024-05-08T15:51:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.052+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fd410015d80 con 0x7fd4200a41b0 2024-05-08T15:51:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.052+0000 7fd4077fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:12.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.052+0000 7fd4077fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:12.310 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37084]: starting. 2024-05-08T15:51:12.310 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37084]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37083 2024-05-08T15:51:12.310 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37084]: created object 0... 2024-05-08T15:51:12.310 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37084]: created object 25... 2024-05-08T15:51:12.311 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37084]: created object 49... 2024-05-08T15:51:12.311 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37084]: finishing. 2024-05-08T15:51:12.311 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37084]: shutting down. 2024-05-08T15:51:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.335+0000 7fd425159640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-05-08T15:51:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.335+0000 7fd425159640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-05-08T15:51:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.335+0000 7fd425159640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:12.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.335+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd400005b80 con 0x7fd4200a41b0 2024-05-08T15:51:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.337+0000 7fd4077fe640 1 -- 172.21.15.7:0/614597006 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7fd410015680 con 0x7fd4200a41b0 2024-05-08T15:51:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.337+0000 7fd4077fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-05-08T15:51:12.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.337+0000 7fd4077fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 >> v1:172.21.15.81:6812/847627118 conn(0x7fd40c03a520 legacy=0x7fd40c03c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 10 monclient: shutdown 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 >> v1:172.21.15.7:6793/0 conn(0x7fd4200a41b0 legacy=0x7fd4201ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 shutdown_connections 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 >> 172.21.15.7:0/614597006 conn(0x7fd4200a9fa0 msgr2=0x7fd420130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 shutdown_connections 2024-05-08T15:51:12.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:12.339+0000 7fd425159640 1 -- 172.21.15.7:0/614597006 wait complete. 2024-05-08T15:51:12.352 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=18,config=2,health=12,kv=10,logm=58} crc {auth=381190772,config=1618278320,health=1535303502,kv=3560234957,logm=1992445723}) 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=16,config=2,health=12,kv=9,logm=61} crc {auth=1389512542,config=1618278320,health=1535303502,kv=1462314268,logm=3734964141}) 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=18,config=2,health=12,kv=9,logm=59} crc {auth=381190772,config=1618278320,health=1535303502,kv=368437079,logm=1626357291}) 2024-05-08T15:51:12.371 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=18,config=2,health=12,kv=9,logm=59} crc {auth=381190772,config=1618278320,health=1535303502,kv=519216810,logm=3095070714}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=18,config=2,health=12,kv=8,logm=60} crc {auth=381190772,config=1618278320,health=1535303502,kv=3287027292,logm=2912071422}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=18,config=2,health=10,kv=9,logm=61} crc {auth=88138375,config=1618278320,health=1264631656,kv=3560234957,logm=1813161392}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=17,config=2,health=10,kv=10,logm=61} crc {auth=475539730,config=1618278320,health=1189910911,kv=3755490522,logm=2131117267}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=15,config=1,health=12,kv=9,logm=63} crc {auth=3984150191,config=1228700967,health=1535303502,kv=368437079,logm=377377329}) 2024-05-08T15:51:12.372 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.337+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=16,config=1,health=11,kv=9,logm=63} crc {auth=939931343,config=2457401934,health=1096256309,kv=313657359,logm=3552271369}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=605836335}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3648112794}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4258653414}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3702513049}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.373 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1638087449}) 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1659785781}) 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4115906776}) 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=38775769}) 2024-05-08T15:51:12.374 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.338+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1449191071}) 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1144016711}) 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3431466575}) 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3948686618}) 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.375 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3419815874}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.339+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2680253066}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3445746429}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4225894638}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3357633659}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.340+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=180325648}) 2024-05-08T15:51:12.376 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.377 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.377 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3584967289}) 2024-05-08T15:51:12.377 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3931189037}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=711232879}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1354005788}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2177838742}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.378 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2348696421}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2527250865}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2849762889}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.341+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3701095245}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1168386661}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3169559877}) 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.379 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1187557771}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2488734998}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4068705460}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3593673951}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1150369188}) 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.380 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3782834546}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.342+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3358767679}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1917030147}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2063875388}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=82747800}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=549702918}) 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.381 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1576556672}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3674605844}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2410131192}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4226083971}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.343+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3370623525}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.344+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=353122836}) 2024-05-08T15:51:12.382 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4079408811}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3042783413}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3613359200}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2499463663}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.383 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1768402381}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3473689986}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3256398017}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.345+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3293216941}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=661676340}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.384 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=712955936}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2245461591}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=522023275}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1415804205}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4106970956}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.385 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1658741993}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2488365616}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.346+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=121769076}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3254961450}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3864649448}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=299574014}) 2024-05-08T15:51:12.386 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3820598984}) 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3204765286}) 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1068163884}) 2024-05-08T15:51:12.387 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.347+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.348+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.348+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4255436367}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.348+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.348+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3317481590}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.348+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2998684272}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2992265815}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1672811374}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.388 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1753757495}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=836319150}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3032388127}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4203764730}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3093609719}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3596661282}) 2024-05-08T15:51:12.389 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.349+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=203502109}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3420405302}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=661916554}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2447038577}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=415823935}) 2024-05-08T15:51:12.390 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=406273242}) 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4012632135}) 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=67442520}) 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.391 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1022113510}) 2024-05-08T15:51:12.392 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.350+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2966368565}) 2024-05-08T15:51:12.392 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.392 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.392 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=668289410}) 2024-05-08T15:51:12.392 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.392 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=81336215}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1317318126}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3191967513}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2185621948}) 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.393 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=336205992}) 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.351+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=225118979}) 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.352+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.352+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=39419236}) 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.352+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1607266095}) 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.394 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2369823838}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=997520173}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1283874951}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1282969835}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.395 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1334353001}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1749802636}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1481394246}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=539958442}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.353+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=647094161}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.396 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=38,mdsmap=3,mgr=7,mgr_command_descs=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=23} crc {logm=3113574836,mdsmap=2840952586,mgr=385272302,mgr_command_descs=3353810608,mgrstat=1321279428,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=4282516149}) 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=38,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=21} crc {logm=4236243986,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810609,mgr_metadata=1815193447,mgrstat=1321279428,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=639130858}) 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=39,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=2,osd_metadata=6,osdmap=22} crc {logm=1497713352,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1321279428,mon_config_key=619004784,monmap=654611407,osd_metadata=1026190808,osdmap=3312118594}) 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.354+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=40,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=7,monmap=3,osd_metadata=4,osd_pg_creating=1,osdmap=24} crc {logm=263686379,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1052327551,mon_config_key=489254662,monmap=256126145,osd_metadata=4240030009,osd_pg_creating=1994635183,osdmap=3563244000}) 2024-05-08T15:51:12.397 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.398 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.398 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=37,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=24} crc {logm=2998273611,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1321279428,mon_config_key=619004784,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=2066006536}) 2024-05-08T15:51:12.398 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.398 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.398 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=34,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=27} crc {logm=1154948886,mdsmap=3140514009,mgr=3460537546,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1321279428,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=3081517083}) 2024-05-08T15:51:12.398 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=39,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=21} crc {logm=3864727864,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1321279428,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=2849613954}) 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=37,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=7,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=26} crc {logm=3674069564,mdsmap=1401145748,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=549302469,mon_config_key=1403364458,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=503369492}) 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.355+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=38,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=22} crc {logm=2189440164,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1321279428,mon_config_key=3508535717,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=1994635183,osdmap=3117336550}) 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.399 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3640512422}) 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1906588180}) 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1591837535}) 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.400 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=463967636}) 2024-05-08T15:51:12.401 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.401 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.401 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=615669542}) 2024-05-08T15:51:12.401 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.401 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.401 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3819427094}) 2024-05-08T15:51:12.401 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.402 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.402 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=234981197}) 2024-05-08T15:51:12.402 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.402 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3162937246}) 2024-05-08T15:51:12.402 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.356+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3215713240}) 2024-05-08T15:51:12.402 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=3983483646}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=3376883297}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.357+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=4270288501}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=492161584}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.403 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1201492023}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=4073537239}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=459352715}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=3467072175}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.358+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2954716929}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=73} crc {osdmap=3728316184}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=72} crc {osdmap=2403565599}) 2024-05-08T15:51:12.404 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=70} crc {osdmap=1569953331}) 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=71} crc {osdmap=2211937451}) 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=73} crc {osdmap=2874042381}) 2024-05-08T15:51:12.405 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=73} crc {osdmap=3466220351}) 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=72} crc {osdmap=3869416223}) 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.359+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=73} crc {osdmap=1475972242}) 2024-05-08T15:51:12.406 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:51:12.360+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=72} crc {osdmap=3240340179}) 2024-05-08T15:51:13.223 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37085]: starting. 2024-05-08T15:51:13.223 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37085]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37083 2024-05-08T15:51:13.223 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37085]: shutting down. 2024-05-08T15:51:13.353 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:51:13.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:51:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.467+0000 7fdbbd5ce640 10 monclient: get_monmap_and_config 2024-05-08T15:51:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.467+0000 7fdbbd5ce640 10 monclient: build_initial_monmap 2024-05-08T15:51:13.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.467+0000 7fdbbd5ce640 10 monclient: monmap: 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:13.468536+0000 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:13.468536+0000 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:13.464 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:13.465 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 1 Processor -- start 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 1 -- start start 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient: init 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient: _add_conns ranks=[7,3,4,0,6,1,5,2,8] 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient(hunting): picked mon.noname-i con 0x7fdbb8137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient(hunting): picked mon.noname-g con 0x7fdbb80a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:13.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient(hunting): picked mon.noname-c con 0x7fdbb80a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb80a2c70 con 0x7fdbb80a49a0 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb80a2f50 con 0x7fdbb80a41b0 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb80a1670 con 0x7fdbb8137940 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbbd5ce640 10 monclient(hunting): authenticate will time out at 1081.583427s 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbb77fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fdbb80a41b0 0x7fdbb8140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55434/0 (socket says 172.21.15.7:55434) 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.468+0000 7fdbb77fe640 1 -- 172.21.15.7:0/4241647062 learned_addr learned my addr 172.21.15.7:0/4241647062 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2086394381 0 0) 0x7fdbb80a2f50 con 0x7fdbb80a41b0 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): my global_id is 114223 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba0003520 con 0x7fdbb80a41b0 2024-05-08T15:51:13.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3379983537 0 0) 0x7fdbb80a1670 con 0x7fdbb8137940 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): my global_id is 114199 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbb80a2f50 con 0x7fdbb8137940 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3919709500 0 0) 0x7fdbb80a2c70 con 0x7fdbb80a49a0 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient(hunting): my global_id is 75909 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbb80a1670 con 0x7fdbb80a49a0 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 613809118 0 0) 0x7fdba0003520 con 0x7fdbb80a41b0 2024-05-08T15:51:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 >> v1:172.21.15.81:6792/0 conn(0x7fdbb8137940 legacy=0x7fdbb8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 >> v1:172.21.15.81:6790/0 conn(0x7fdbb80a49a0 legacy=0x7fdbb813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 monclient: found mon.noname-c 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb80a39f0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: _finish_auth 0 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdbb0001b30 con 0x7fdbb80a41b0 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbbd5ce640 5 monclient: authenticate success, global_id 114223 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbbd5ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: dump: 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbbd5ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdbb0001e50 con 0x7fdbb80a41b0 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 1 -- 172.21.15.7:0/4241647062 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdbb0004f60 con 0x7fdbb80a41b0 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbbd5ce640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbbd5ce640 4 set_mon_vals no callback set 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.469+0000 7fdbb5ffb640 10 monclient: dump: 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 10 monclient: shutdown 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/4241647062 >> v1:172.21.15.7:6791/0 conn(0x7fdbb80a41b0 legacy=0x7fdbb8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/4241647062 shutdown_connections 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/4241647062 >> 172.21.15.7:0/4241647062 conn(0x7fdbb80a9fa0 msgr2=0x7fdbb80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/4241647062 shutdown_connections 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/4241647062 wait complete. 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 10 monclient: build_initial_monmap 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.470+0000 7fdbbd5ce640 10 monclient: monmap: 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 1 Processor -- start 2024-05-08T15:51:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 1 -- start start 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient: init 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient: _add_conns ranks=[2,8,4,1,0,6,7,3,5] 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient(hunting): picked mon.noname-b con 0x7fdbb80a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient(hunting): picked mon.noname-e con 0x7fdbb80a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient(hunting): picked mon.noname-c con 0x7fdbb8137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb8136af0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb80a0f60 con 0x7fdbb8137940 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbb81330a0 con 0x7fdbb80a49a0 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbbd5ce640 10 monclient(hunting): authenticate will time out at 1081.586402s 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbb77fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fdbb8137940 0x7fdbb81d4e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55448/0 (socket says 172.21.15.7:55448) 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbb77fe640 1 -- 172.21.15.7:0/1258524743 learned_addr learned my addr 172.21.15.7:0/1258524743 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbb6ffd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fdbb80a41b0 0x7fdbb8132280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35386/0 (socket says 172.21.15.7:35386) 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.471+0000 7fdbb67fc640 1 --1- 172.21.15.7:0/1258524743 >> v1:172.21.15.7:6793/0 conn(0x7fdbb80a49a0 0x7fdbb8132990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50438/0 (socket says 172.21.15.7:50438) 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463242621 0 0) 0x7fdbb80a39f0 con 0x7fdbb8137940 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): my global_id is 114232 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdba4003520 con 0x7fdbb8137940 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559585812 0 0) 0x7fdbb80a2f50 con 0x7fdbb80a41b0 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): my global_id is 95015 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbb80a39f0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3086812617 0 0) 0x7fdbb81330a0 con 0x7fdbb80a49a0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): my global_id is 94877 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbb80a2f50 con 0x7fdbb80a49a0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2104690669 0 0) 0x7fdbb80a39f0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdbb81330a0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3461002532 0 0) 0x7fdbb80a2f50 con 0x7fdbb80a49a0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdbb80a39f0 con 0x7fdbb80a49a0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2306359060 0 0) 0x7fdba4003520 con 0x7fdbb8137940 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fdbb80a2f50 con 0x7fdbb8137940 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdba80018d0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): dump: 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.484 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdbac0019d0 con 0x7fdbb80a49a0 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): dump: 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdbb0004ea0 con 0x7fdbb8137940 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient(hunting): dump: 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3213645417 0 0) 0x7fdbb81330a0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 >> v1:172.21.15.7:6793/0 conn(0x7fdbb80a49a0 legacy=0x7fdbb8132990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 >> v1:172.21.15.7:6791/0 conn(0x7fdbb8137940 legacy=0x7fdbb81d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 monclient: found mon.b 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb8133c70 con 0x7fdbb80a41b0 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbbd5ce640 5 monclient: authenticate success, global_id 95015 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbbd5ce640 10 monclient: _renew_subs 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbbd5ce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb80a1f30 con 0x7fdbb80a41b0 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbbd5ce640 10 monclient: _renew_subs 2024-05-08T15:51:13.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbbd5ce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fdbb8136360 con 0x7fdbb80a41b0 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fdba8001d00 con 0x7fdbb80a41b0 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fdba8003c00 con 0x7fdbb80a41b0 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdb9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.472+0000 7fdbb57fa640 4 set_mon_vals no callback set 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.473+0000 7fdb9f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.473+0000 7fdb9f7fe640 10 monclient: dump: 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.473+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fdba8003e20 con 0x7fdbb80a41b0 2024-05-08T15:51:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.473+0000 7fdbbd5ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.473+0000 7fdbbd5ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.473+0000 7fdbbd5ce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.473+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb98005080 con 0x7fdbb80a41b0 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.474+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(196..196 src has 1..196) ==== 3506+0+0 (unknown 209529885 0 0) 0x7fdba8046c60 con 0x7fdbb80a41b0 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.474+0000 7fdbb67fc640 10 monclient: get_auth_request con 0x7fdba403a2f0 auth_method 0 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.478+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fdbb8136af0 con 0x7fdbb80a41b0 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.478+0000 7fdb9f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.478+0000 7fdb9f7fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.732+0000 7fdbbd5ce640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:51:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.732+0000 7fdbbd5ce640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.732+0000 7fdbbd5ce640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.732+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdb98005370 con 0x7fdbb80a41b0 2024-05-08T15:51:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.732+0000 7fdb9f7fe640 1 -- 172.21.15.7:0/1258524743 <== mon.2 v1:172.21.15.7:6790/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 551961435) 0x7fdba8015e50 con 0x7fdbb80a41b0 2024-05-08T15:51:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.732+0000 7fdb9f7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.732+0000 7fdb9f7fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:13.729 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":28,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:51:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 >> v1:172.21.15.81:6812/847627118 conn(0x7fdba403a2f0 legacy=0x7fdba403c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 10 monclient: shutdown 2024-05-08T15:51:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 >> v1:172.21.15.7:6790/0 conn(0x7fdbb80a41b0 legacy=0x7fdbb8132280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 shutdown_connections 2024-05-08T15:51:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 >> 172.21.15.7:0/1258524743 conn(0x7fdbb80a9fa0 msgr2=0x7fdbb80a8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:13.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 shutdown_connections 2024-05-08T15:51:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.734+0000 7fdbbd5ce640 1 -- 172.21.15.7:0/1258524743 wait complete. 2024-05-08T15:51:13.741 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:51:13.741 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T15:51:13.741 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:51:13.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:51:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.849+0000 7efcda43f640 10 monclient: get_monmap_and_config 2024-05-08T15:51:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.849+0000 7efcda43f640 10 monclient: build_initial_monmap 2024-05-08T15:51:13.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.849+0000 7efcda43f640 10 monclient: monmap: 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:13.850641+0000 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:13.850641+0000 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:13.846 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:13.847 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:13.847 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 1 Processor -- start 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 1 -- start start 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient: init 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient: _add_conns ranks=[1,7,5,6,0,4,8,2,3] 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient(hunting): picked mon.noname-f con 0x7efcd40a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient(hunting): picked mon.noname-i con 0x7efcd40a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient(hunting): picked mon.noname-h con 0x7efcd413e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:13.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd40a1660 con 0x7efcd40a53a0 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd40a28f0 con 0x7efcd413e310 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd40a2bd0 con 0x7efcd40a5780 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.850+0000 7efcda43f640 10 monclient(hunting): authenticate will time out at 1081.965578s 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd37fe640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7efcd40a5780 0x7efcd413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:48636/0 (socket says 172.21.15.7:48636) 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd89dc640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7efcd413e310 0x7efcd4140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:54056/0 (socket says 172.21.15.7:54056) 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd37fe640 1 -- 172.21.15.7:0/4075932537 learned_addr learned my addr 172.21.15.7:0/4075932537 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2152911689 0 0) 0x7efcd40a2bd0 con 0x7efcd40a5780 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): my global_id is 114208 2024-05-08T15:51:13.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcbc003520 con 0x7efcd40a5780 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150436008 0 0) 0x7efcd40a1660 con 0x7efcd40a53a0 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): my global_id is 5617 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcd40a2bd0 con 0x7efcd40a53a0 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1717353117 0 0) 0x7efcd40a28f0 con 0x7efcd413e310 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient(hunting): my global_id is 5657 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcd40a1660 con 0x7efcd413e310 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1725012939 0 0) 0x7efcbc003520 con 0x7efcd40a5780 2024-05-08T15:51:13.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 >> v1:172.21.15.81:6791/0 conn(0x7efcd413e310 legacy=0x7efcd4140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 >> v1:172.21.15.81:6789/0 conn(0x7efcd40a53a0 legacy=0x7efcd40a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 monclient: found mon.noname-i 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.81:6792/0 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd40a6730 con 0x7efcd40a5780 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcc80016e0 con 0x7efcd40a5780 2024-05-08T15:51:13.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:51:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: dump: 2024-05-08T15:51:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.861 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.863 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcda43f640 5 monclient: authenticate success, global_id 114208 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcda43f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 <== mon.7 v1:172.21.15.81:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efcc8001a00 con 0x7efcd40a5780 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 1 -- 172.21.15.7:0/4075932537 <== mon.7 v1:172.21.15.81:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcc8004be0 con 0x7efcd40a5780 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcda43f640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcda43f640 4 set_mon_vals no callback set 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.851+0000 7efcd2ffd640 10 monclient: dump: 2024-05-08T15:51:13.864 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.865 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.852+0000 7efcda43f640 10 monclient: shutdown 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.852+0000 7efcda43f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.852+0000 7efcda43f640 1 -- 172.21.15.7:0/4075932537 >> v1:172.21.15.81:6792/0 conn(0x7efcd40a5780 legacy=0x7efcd413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.852+0000 7efcda43f640 1 -- 172.21.15.7:0/4075932537 shutdown_connections 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.852+0000 7efcda43f640 1 -- 172.21.15.7:0/4075932537 >> 172.21.15.7:0/4075932537 conn(0x7efcd40a9fa0 msgr2=0x7efcd40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.852+0000 7efcda43f640 1 -- 172.21.15.7:0/4075932537 shutdown_connections 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.852+0000 7efcda43f640 1 -- 172.21.15.7:0/4075932537 wait complete. 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.853+0000 7efcda43f640 10 monclient: build_initial_monmap 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.853+0000 7efcda43f640 10 monclient: monmap: 2024-05-08T15:51:13.866 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:13.867 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:13.868 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.853+0000 7efcda43f640 1 Processor -- start 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.853+0000 7efcda43f640 1 -- start start 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.853+0000 7efcda43f640 10 monclient: init 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 10 monclient: _add_conns ranks=[0,6,2,7,4,1,3,8,5] 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 10 monclient(hunting): picked mon.noname-a con 0x7efcd40a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 10 monclient(hunting): picked mon.noname-d con 0x7efcd40a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 10 monclient(hunting): picked mon.noname-b con 0x7efcd413e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:13.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd4135120 con 0x7efcd40a53a0 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd4134e40 con 0x7efcd413e310 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd4135400 con 0x7efcd40a5780 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcda43f640 10 monclient(hunting): authenticate will time out at 1081.968816s 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd3fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efcd40a53a0 0x7efcd4137920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49752/0 (socket says 172.21.15.7:49752) 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd89dc640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7efcd413e310 0x7efcd41d57f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35398/0 (socket says 172.21.15.7:35398) 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd37fe640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7efcd40a5780 0x7efcd41d20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:46906/0 (socket says 172.21.15.7:46906) 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd3fff640 1 -- 172.21.15.7:0/1041089894 learned_addr learned my addr 172.21.15.7:0/1041089894 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089027874 0 0) 0x7efcd4134e40 con 0x7efcd413e310 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): my global_id is 95024 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcc0003520 con 0x7efcd413e310 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3383252811 0 0) 0x7efcd4135400 con 0x7efcd40a5780 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): my global_id is 75723 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcd4134e40 con 0x7efcd40a5780 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472488710 0 0) 0x7efcd40a2bd0 con 0x7efcd40a53a0 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): my global_id is 94887 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcd4135400 con 0x7efcd40a53a0 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1180052484 0 0) 0x7efcc0003520 con 0x7efcd413e310 2024-05-08T15:51:13.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efcd40a2bd0 con 0x7efcd413e310 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efccc001ba0 con 0x7efcd413e310 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.872 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 479356500 0 0) 0x7efcd4135400 con 0x7efcd40a53a0 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efcc0003520 con 0x7efcd40a53a0 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.854+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2646721625 0 0) 0x7efcd4134e40 con 0x7efcd40a5780 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efcd4135400 con 0x7efcd40a5780 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcc4001950 con 0x7efcd40a53a0 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:13.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.874 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcc8004b20 con 0x7efcd40a5780 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.875 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3384188852 0 0) 0x7efcd40a2bd0 con 0x7efcd413e310 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.7:6792/0 conn(0x7efcd40a5780 legacy=0x7efcd41d20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.7:6789/0 conn(0x7efcd40a53a0 legacy=0x7efcd4137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 monclient: found mon.b 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd413d020 con 0x7efcd413e310 2024-05-08T15:51:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: _finish_auth 0 2024-05-08T15:51:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcda43f640 5 monclient: authenticate success, global_id 95024 2024-05-08T15:51:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcda43f640 10 monclient: _renew_subs 2024-05-08T15:51:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcda43f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7efcd41cd290 con 0x7efcd413e310 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcda43f640 10 monclient: _renew_subs 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcda43f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7efcd41cd580 con 0x7efcd413e310 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efccc004b60 con 0x7efcd413e310 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efccc005050 con 0x7efcd413e310 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd27fc640 4 set_mon_vals no callback set 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.855+0000 7efcd0ff9640 10 monclient: dump: 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.878 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.879 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcda43f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efcb40065c0 con 0x7efcb4002860 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7efccc0103d0 con 0x7efcd413e310 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(196..196 src has 1..196) ==== 3506+0+0 (unknown 209529885 0 0) 0x7efccc046eb0 con 0x7efcd413e310 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2913296484 0 0) 0x7efcb40065c0 con 0x7efcb4002860 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 10 monclient(hunting): my global_id is 94896 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcd40a2bd0 con 0x7efcb4002860 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3382912459 0 0) 0x7efcd40a2bd0 con 0x7efcb4002860 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efcb40065c0 con 0x7efcb4002860 2024-05-08T15:51:13.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcc4003950 con 0x7efcb4002860 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 10 monclient: dump: 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.881 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd37fe640 10 monclient: get_auth_request con 0x7efcc003a4e0 auth_method 0 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 866698538 0 0) 0x7efcb40065c0 con 0x7efcb4002860 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.856+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcb400a150 con 0x7efcb4002860 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.857+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7efcc40057c0 con 0x7efcb4002860 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.857+0000 7efcd0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.857+0000 7efcd0ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.857+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.7:6789/0 conn(0x7efcb4002860 legacy=0x7efcb4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.857+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.7:6789/0 conn(0x7efcb4002860 legacy=0x7efcb4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcda43f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efcb4009e20 con 0x7efcb40090e0 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1857046461 0 0) 0x7efcb4009e20 con 0x7efcb40090e0 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcd0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcd0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcd0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcd0ff9640 10 monclient(hunting): my global_id is 94905 2024-05-08T15:51:13.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.863+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcb40065c0 con 0x7efcb40090e0 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 46364124 0 0) 0x7efcb40065c0 con 0x7efcb40090e0 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efcb4009e20 con 0x7efcb40090e0 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcc40041c0 con 0x7efcb40090e0 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 10 monclient: dump: 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:13.885 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 299442636 0 0) 0x7efcb4009e20 con 0x7efcb40090e0 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efcb400c390 con 0x7efcb40090e0 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 889257325) 0x7efcc4005970 con 0x7efcb40090e0 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.7:6789/0 conn(0x7efcb40090e0 legacy=0x7efcb4009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd0ff9640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.7:6789/0 conn(0x7efcb40090e0 legacy=0x7efcb4009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:13.887 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:13.887 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 124415, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:13.888 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 3, 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:13.889 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:13.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:13.891 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:13.892 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.894 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:13.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:13.901 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.902 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:13.903 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.904 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:13.905 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:13.906 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:13.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:13.908 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:13.908 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.81:6812/847627118 conn(0x7efcc003a4e0 legacy=0x7efcc003c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcda43f640 10 monclient: shutdown 2024-05-08T15:51:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcda43f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 >> v1:172.21.15.7:6790/0 conn(0x7efcd413e310 legacy=0x7efcd41d57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd89dc640 1 -- 172.21.15.7:0/1041089894 reap_dead start 2024-05-08T15:51:13.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcd89dc640 1 -- 172.21.15.7:0/1041089894 reap_dead start 2024-05-08T15:51:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 shutdown_connections 2024-05-08T15:51:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.864+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 >> 172.21.15.7:0/1041089894 conn(0x7efcd40a9fa0 msgr2=0x7efcd40a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.865+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 shutdown_connections 2024-05-08T15:51:13.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:13.865+0000 7efcda43f640 1 -- 172.21.15.7:0/1041089894 wait complete. 2024-05-08T15:51:13.911 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:13.911 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:51:13.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:51:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.019+0000 7fca9ab8f640 10 monclient: get_monmap_and_config 2024-05-08T15:51:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.019+0000 7fca9ab8f640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.019+0000 7fca9ab8f640 10 monclient: monmap: 2024-05-08T15:51:14.015 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:14.020682+0000 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:14.020682+0000 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.016 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.017 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.020+0000 7fca9ab8f640 1 Processor -- start 2024-05-08T15:51:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.020+0000 7fca9ab8f640 1 -- start start 2024-05-08T15:51:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.020+0000 7fca9ab8f640 10 monclient: init 2024-05-08T15:51:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.020+0000 7fca9ab8f640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.020+0000 7fca9ab8f640 10 monclient: _add_conns ranks=[7,4,1,3,8,2,0,5,6] 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 10 monclient(hunting): picked mon.noname-i con 0x7fca94137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 10 monclient(hunting): picked mon.noname-c con 0x7fca940a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 10 monclient(hunting): picked mon.noname-f con 0x7fca940a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca940a2c70 con 0x7fca940a41b0 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca940a2f50 con 0x7fca940a49a0 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca940a1670 con 0x7fca94137940 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 10 monclient(hunting): authenticate will time out at 1082.135735s 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca93fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fca940a49a0 0x7fca9413cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55450/0 (socket says 172.21.15.7:55450) 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca93fff640 1 -- 172.21.15.7:0/2549690091 learned_addr learned my addr 172.21.15.7:0/2549690091 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 900785699 0 0) 0x7fca940a2f50 con 0x7fca940a49a0 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): my global_id is 114241 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca84003520 con 0x7fca940a49a0 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535474928 0 0) 0x7fca940a1670 con 0x7fca94137940 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): my global_id is 114217 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca940a2f50 con 0x7fca94137940 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907075479 0 0) 0x7fca940a2c70 con 0x7fca940a41b0 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient(hunting): my global_id is 5626 2024-05-08T15:51:14.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca940a1670 con 0x7fca940a41b0 2024-05-08T15:51:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 656870246 0 0) 0x7fca84003520 con 0x7fca940a49a0 2024-05-08T15:51:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 >> v1:172.21.15.81:6792/0 conn(0x7fca94137940 legacy=0x7fca94137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 >> v1:172.21.15.81:6789/0 conn(0x7fca940a41b0 legacy=0x7fca94140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 monclient: found mon.noname-c 2024-05-08T15:51:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca940a39f0 con 0x7fca940a49a0 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca800017b0 con 0x7fca940a49a0 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 5 monclient: authenticate success, global_id 114241 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: dump: 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.032 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.034 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.034 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fca80001ad0 con 0x7fca940a49a0 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 1 -- 172.21.15.7:0/2549690091 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca80004be0 con 0x7fca940a49a0 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca9ab8f640 4 set_mon_vals no callback set 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.021+0000 7fca937fe640 10 monclient: dump: 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.035 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.036 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 10 monclient: shutdown 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 1 -- 172.21.15.7:0/2549690091 >> v1:172.21.15.7:6791/0 conn(0x7fca940a49a0 legacy=0x7fca9413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 1 -- 172.21.15.7:0/2549690091 shutdown_connections 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 1 -- 172.21.15.7:0/2549690091 >> 172.21.15.7:0/2549690091 conn(0x7fca940a9fa0 msgr2=0x7fca940aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 1 -- 172.21.15.7:0/2549690091 shutdown_connections 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 1 -- 172.21.15.7:0/2549690091 wait complete. 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.022+0000 7fca9ab8f640 10 monclient: monmap: 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:14.037 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.023+0000 7fca9ab8f640 1 Processor -- start 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.023+0000 7fca9ab8f640 1 -- start start 2024-05-08T15:51:14.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.023+0000 7fca9ab8f640 10 monclient: init 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.023+0000 7fca9ab8f640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.023+0000 7fca9ab8f640 10 monclient: _add_conns ranks=[0,8,7,5,1,2,4,6,3] 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.023+0000 7fca9ab8f640 10 monclient(hunting): picked mon.noname-a con 0x7fca940a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 10 monclient(hunting): picked mon.noname-e con 0x7fca940a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 10 monclient(hunting): picked mon.noname-i con 0x7fca94137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca940a1f30 con 0x7fca940a41b0 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca94135a40 con 0x7fca94137940 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca940a0f60 con 0x7fca940a49a0 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 10 monclient(hunting): authenticate will time out at 1082.138710s 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9892b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fca940a41b0 0x7fca94131e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49780/0 (socket says 172.21.15.7:49780) 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9892b640 1 -- 172.21.15.7:0/3093928275 learned_addr learned my addr 172.21.15.7:0/3093928275 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230338070 0 0) 0x7fca940a2f50 con 0x7fca940a41b0 2024-05-08T15:51:14.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): my global_id is 94914 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca78003520 con 0x7fca940a41b0 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4018734386 0 0) 0x7fca940a0f60 con 0x7fca940a49a0 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): my global_id is 94886 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca940a2f50 con 0x7fca940a49a0 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4008675448 0 0) 0x7fca94135a40 con 0x7fca94137940 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): my global_id is 114226 2024-05-08T15:51:14.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca940a0f60 con 0x7fca94137940 2024-05-08T15:51:14.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 232662201 0 0) 0x7fca940a2f50 con 0x7fca940a49a0 2024-05-08T15:51:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca94135a40 con 0x7fca940a49a0 2024-05-08T15:51:14.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 132824510 0 0) 0x7fca940a0f60 con 0x7fca94137940 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca940a2f50 con 0x7fca94137940 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1436809104 0 0) 0x7fca78003520 con 0x7fca940a41b0 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca940a0f60 con 0x7fca940a41b0 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca80004b20 con 0x7fca940a49a0 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): dump: 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.043 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.044 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca8c001d50 con 0x7fca94137940 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): dump: 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.045 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.046 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca88001970 con 0x7fca940a41b0 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient(hunting): dump: 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.047 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 916449056 0 0) 0x7fca94135a40 con 0x7fca940a49a0 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.81:6792/0 conn(0x7fca94137940 legacy=0x7fca941d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.7:6789/0 conn(0x7fca940a41b0 legacy=0x7fca94131e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 monclient: found mon.e 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca94132c60 con 0x7fca940a49a0 2024-05-08T15:51:14.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca917fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 5 monclient: authenticate success, global_id 94886 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 10 monclient: _renew_subs 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.024+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fca94135760 con 0x7fca940a49a0 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca9ab8f640 10 monclient: _renew_subs 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca9ab8f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fca94134590 con 0x7fca940a49a0 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fca80001850 con 0x7fca940a49a0 2024-05-08T15:51:14.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca917fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca800051a0 con 0x7fca940a49a0 2024-05-08T15:51:14.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca92ffd640 4 set_mon_vals no callback set 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca917fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca917fa640 10 monclient: dump: 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.051 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.052 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.052 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.052 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.052 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.052 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca9ab8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fca740065c0 con 0x7fca74002860 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.025+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fca80010520 con 0x7fca940a49a0 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(196..196 src has 1..196) ==== 3506+0+0 (unknown 209529885 0 0) 0x7fca80046d50 con 0x7fca940a49a0 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794845700 0 0) 0x7fca740065c0 con 0x7fca74002860 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient(hunting): my global_id is 95033 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca94135a40 con 0x7fca74002860 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca9912c640 10 monclient: get_auth_request con 0x7fca7803a290 auth_method 0 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4141525194 0 0) 0x7fca94135a40 con 0x7fca74002860 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca740065c0 con 0x7fca74002860 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca88003930 con 0x7fca74002860 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient: dump: 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.054 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2960223749 0 0) 0x7fca740065c0 con 0x7fca74002860 2024-05-08T15:51:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fca7400a150 con 0x7fca74002860 2024-05-08T15:51:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fca880057c0 con 0x7fca74002860 2024-05-08T15:51:14.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.7:6790/0 conn(0x7fca74002860 legacy=0x7fca74004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.026+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.7:6790/0 conn(0x7fca74002860 legacy=0x7fca74004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca9ab8f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fca74009e20 con 0x7fca740090e0 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 932064122 0 0) 0x7fca74009e20 con 0x7fca740090e0 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca917fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca917fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca917fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca917fa640 10 monclient(hunting): my global_id is 95042 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.033+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fca740065c0 con 0x7fca740090e0 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3261997869 0 0) 0x7fca740065c0 con 0x7fca740090e0 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fca74009e20 con 0x7fca740090e0 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fca88004250 con 0x7fca740090e0 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 10 monclient: dump: 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 780597556 0 0) 0x7fca74009e20 con 0x7fca740090e0 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fca7400c390 con 0x7fca740090e0 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 2260738237) 0x7fca88005970 con 0x7fca740090e0 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.7:6790/0 conn(0x7fca740090e0 legacy=0x7fca74009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca917fa640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.7:6790/0 conn(0x7fca740090e0 legacy=0x7fca74009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:14.061 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 124585, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 3, 2024-05-08T15:51:14.062 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:14.063 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.064 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.065 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:14.066 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.067 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.068 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:14.069 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.070 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.071 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.072 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:14.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.076 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.077 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.079 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:14.080 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.81:6812/847627118 conn(0x7fca7803a290 legacy=0x7fca7803c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca9ab8f640 10 monclient: shutdown 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca9ab8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 >> v1:172.21.15.7:6793/0 conn(0x7fca940a49a0 legacy=0x7fca94132550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca9912c640 1 -- 172.21.15.7:0/3093928275 reap_dead start 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.034+0000 7fca9912c640 1 -- 172.21.15.7:0/3093928275 reap_dead start 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.035+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 shutdown_connections 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.035+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 >> 172.21.15.7:0/3093928275 conn(0x7fca940a9fa0 msgr2=0x7fca941308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.035+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 shutdown_connections 2024-05-08T15:51:14.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.035+0000 7fca9ab8f640 1 -- 172.21.15.7:0/3093928275 wait complete. 2024-05-08T15:51:14.083 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:14.083 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:51:14.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.192+0000 7fa0aa6f2640 10 monclient: get_monmap_and_config 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.192+0000 7fa0aa6f2640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.192+0000 7fa0aa6f2640 10 monclient: monmap: 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:14.193072+0000 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:14.193072+0000 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.188 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.189 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.189 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.192+0000 7fa0aa6f2640 1 Processor -- start 2024-05-08T15:51:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.192+0000 7fa0aa6f2640 1 -- start start 2024-05-08T15:51:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.192+0000 7fa0aa6f2640 10 monclient: init 2024-05-08T15:51:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 10 monclient: _add_conns ranks=[7,0,3,6,5,2,4,8,1] 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 10 monclient(hunting): picked mon.noname-i con 0x7fa0a40a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 10 monclient(hunting): picked mon.noname-a con 0x7fa0a40a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 10 monclient(hunting): picked mon.noname-g con 0x7fa0a413e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a40a1660 con 0x7fa0a40a5780 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a40a28f0 con 0x7fa0a413e310 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a40a2bd0 con 0x7fa0a40a53a0 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0aa6f2640 10 monclient(hunting): authenticate will time out at 1082.308015s 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a37fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0a40a5780 0x7fa0a413cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49784/0 (socket says 172.21.15.7:49784) 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a37fe640 1 -- 172.21.15.7:0/1755991861 learned_addr learned my addr 172.21.15.7:0/1755991861 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3268104074 0 0) 0x7fa0a40a1660 con 0x7fa0a40a5780 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): my global_id is 94923 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa08c003520 con 0x7fa0a40a5780 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809355655 0 0) 0x7fa0a40a28f0 con 0x7fa0a413e310 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): my global_id is 75918 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0a40a1660 con 0x7fa0a413e310 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1419617902 0 0) 0x7fa0a40a2bd0 con 0x7fa0a40a53a0 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 10 monclient(hunting): my global_id is 114235 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0a40a28f0 con 0x7fa0a40a53a0 2024-05-08T15:51:14.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.193+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1086669391 0 0) 0x7fa08c003520 con 0x7fa0a40a5780 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 >> v1:172.21.15.81:6792/0 conn(0x7fa0a40a53a0 legacy=0x7fa0a40a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 >> v1:172.21.15.81:6790/0 conn(0x7fa0a413e310 legacy=0x7fa0a4140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 1 monclient: found mon.noname-a 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40a6730 con 0x7fa0a40a5780 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 5 monclient: authenticate success, global_id 94923 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa0980017b0 con 0x7fa0a40a5780 2024-05-08T15:51:14.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: dump: 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.203 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.204 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa098001ad0 con 0x7fa0a40a5780 2024-05-08T15:51:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 1 -- 172.21.15.7:0/1755991861 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa098004be0 con 0x7fa0a40a5780 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 4 set_mon_vals no callback set 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0a2ffd640 10 monclient: dump: 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.205 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 10 monclient: shutdown 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1755991861 >> v1:172.21.15.7:6789/0 conn(0x7fa0a40a5780 legacy=0x7fa0a413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1755991861 shutdown_connections 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1755991861 >> 172.21.15.7:0/1755991861 conn(0x7fa0a40a9fa0 msgr2=0x7fa0a40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1755991861 shutdown_connections 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.194+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1755991861 wait complete. 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.195+0000 7fa0aa6f2640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.195+0000 7fa0aa6f2640 10 monclient: monmap: 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.207 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.195+0000 7fa0aa6f2640 1 Processor -- start 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.195+0000 7fa0aa6f2640 1 -- start start 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.195+0000 7fa0aa6f2640 10 monclient: init 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 10 monclient: _add_conns ranks=[5,8,3,1,0,6,2,4,7] 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 10 monclient(hunting): picked mon.noname-h con 0x7fa0a40a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 10 monclient(hunting): picked mon.noname-e con 0x7fa0a40a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 10 monclient(hunting): picked mon.noname-g con 0x7fa0a413e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a4134c40 con 0x7fa0a413e310 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a41cdb30 con 0x7fa0a40a53a0 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0a40a1e00 con 0x7fa0a40a5780 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0aa6f2640 10 monclient(hunting): authenticate will time out at 1082.310949s 2024-05-08T15:51:14.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a37fe640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fa0a40a5780 0x7fa0a41cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50454/0 (socket says 172.21.15.7:50454) 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a37fe640 1 -- 172.21.15.7:0/1528396955 learned_addr learned my addr 172.21.15.7:0/1528396955 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751087942 0 0) 0x7fa0a40a1e00 con 0x7fa0a40a5780 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): my global_id is 94895 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa090003520 con 0x7fa0a40a5780 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804736596 0 0) 0x7fa0a41cdb30 con 0x7fa0a40a53a0 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): my global_id is 5666 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0a40a1e00 con 0x7fa0a40a53a0 2024-05-08T15:51:14.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349609396 0 0) 0x7fa0a4134c40 con 0x7fa0a413e310 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): my global_id is 75927 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0a41cdb30 con 0x7fa0a413e310 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 162873941 0 0) 0x7fa090003520 con 0x7fa0a40a5780 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa0a4134c40 con 0x7fa0a40a5780 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa098004b20 con 0x7fa0a40a5780 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.211 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.196+0000 7fa0a0ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 206059757 0 0) 0x7fa0a41cdb30 con 0x7fa0a413e310 2024-05-08T15:51:14.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa090003520 con 0x7fa0a413e310 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1642983742 0 0) 0x7fa0a40a1e00 con 0x7fa0a40a53a0 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa0a41cdb30 con 0x7fa0a40a53a0 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa09c001b40 con 0x7fa0a413e310 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.213 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.214 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa094001990 con 0x7fa0a40a53a0 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.215 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2994663178 0 0) 0x7fa0a4134c40 con 0x7fa0a40a5780 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.81:6791/0 conn(0x7fa0a40a53a0 legacy=0x7fa0a41ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.81:6790/0 conn(0x7fa0a413e310 legacy=0x7fa0a41d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 monclient: found mon.e 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a40a43c0 con 0x7fa0a40a5780 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 5 monclient: authenticate success, global_id 94895 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 10 monclient: _renew_subs 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a4134960 con 0x7fa0a40a5780 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 10 monclient: _renew_subs 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fa0a4133a60 con 0x7fa0a40a5780 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa098001ad0 con 0x7fa0a40a5780 2024-05-08T15:51:14.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa098001d00 con 0x7fa0a40a5780 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a1ffb640 4 set_mon_vals no callback set 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 10 monclient: dump: 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.218 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0aa6f2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.197+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa0980106c0 con 0x7fa0a40a5780 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa0840065c0 con 0x7fa084002860 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(196..196 src has 1..196) ==== 3506+0+0 (unknown 209529885 0 0) 0x7fa098046dc0 con 0x7fa0a40a5780 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2742272034 0 0) 0x7fa0840065c0 con 0x7fa084002860 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 10 monclient(hunting): my global_id is 114250 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0a4134c40 con 0x7fa084002860 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a8c8f640 10 monclient: get_auth_request con 0x7fa09003a500 auth_method 0 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1145935453 0 0) 0x7fa0a4134c40 con 0x7fa084002860 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa0840065c0 con 0x7fa084002860 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa094003950 con 0x7fa084002860 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 10 monclient: dump: 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.220 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.221 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2672637577 0 0) 0x7fa0840065c0 con 0x7fa084002860 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.198+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa08400a150 con 0x7fa084002860 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.199+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fa094003d70 con 0x7fa084002860 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.199+0000 7fa0a0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.199+0000 7fa0a0ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.199+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.7:6791/0 conn(0x7fa084002860 legacy=0x7fa084004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.199+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.7:6791/0 conn(0x7fa084002860 legacy=0x7fa084004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0aa6f2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:14.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fa084009e40 con 0x7fa084009100 2024-05-08T15:51:14.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844644939 0 0) 0x7fa084009e40 con 0x7fa084009100 2024-05-08T15:51:14.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0a0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0a0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0a0ff9640 10 monclient(hunting): my global_id is 114259 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.205+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0840065c0 con 0x7fa084009100 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 686330845 0 0) 0x7fa0840065c0 con 0x7fa084009100 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa084009e40 con 0x7fa084009100 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa094004640 con 0x7fa084009100 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 10 monclient: dump: 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.226 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2466302479 0 0) 0x7fa084009e40 con 0x7fa084009100 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa08400c3d0 con 0x7fa084009100 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7521 (unknown 0 0 3421239740) 0x7fa0940047d0 con 0x7fa084009100 2024-05-08T15:51:14.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.7:6791/0 conn(0x7fa084009100 legacy=0x7fa084009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a0ff9640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.7:6791/0 conn(0x7fa084009100 legacy=0x7fa084009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 9050, 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:14.228 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.229 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.230 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:14.231 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.232 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.233 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.234 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.236 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.237 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.241 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.243 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:14.244 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.245 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.246 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.247 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.248 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.249 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.250 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:14.250 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.81:6812/847627118 conn(0x7fa09003a500 legacy=0x7fa09003c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 10 monclient: shutdown 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 >> v1:172.21.15.7:6793/0 conn(0x7fa0a40a5780 legacy=0x7fa0a41cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a8c8f640 1 -- 172.21.15.7:0/1528396955 reap_dead start 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0a8c8f640 1 -- 172.21.15.7:0/1528396955 reap_dead start 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 shutdown_connections 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 >> 172.21.15.7:0/1528396955 conn(0x7fa0a40a9fa0 msgr2=0x7fa0a40a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 shutdown_connections 2024-05-08T15:51:14.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.206+0000 7fa0aa6f2640 1 -- 172.21.15.7:0/1528396955 wait complete. 2024-05-08T15:51:14.252 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:14.252 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:51:14.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.362+0000 7f74818cf640 10 monclient: get_monmap_and_config 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.362+0000 7f74818cf640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.362+0000 7f74818cf640 10 monclient: monmap: 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:14.363254+0000 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:14.363254+0000 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.358 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.359 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.362+0000 7f74818cf640 1 Processor -- start 2024-05-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.362+0000 7f74818cf640 1 -- start start 2024-05-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.362+0000 7f74818cf640 10 monclient: init 2024-05-08T15:51:14.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 10 monclient: _add_conns ranks=[1,3,0,4,2,7,8,5,6] 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 10 monclient(hunting): picked mon.noname-f con 0x7f747c0a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 10 monclient(hunting): picked mon.noname-g con 0x7f747c0a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 10 monclient(hunting): picked mon.noname-a con 0x7f747c13e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0a1660 con 0x7f747c13e310 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0a28f0 con 0x7f747c0a53a0 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0a2bd0 con 0x7f747c0a5780 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f74818cf640 10 monclient(hunting): authenticate will time out at 1082.478168s 2024-05-08T15:51:14.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f747b7fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f747c13e310 0x7f747c140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:49786/0 (socket says 172.21.15.7:49786) 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f747b7fe640 1 -- 172.21.15.7:0/1573970457 learned_addr learned my addr 172.21.15.7:0/1573970457 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4292580329 0 0) 0x7f747c0a1660 con 0x7f747c13e310 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f7479ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f7479ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f7479ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f7479ffb640 10 monclient(hunting): my global_id is 94932 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.363+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7468003520 con 0x7f747c13e310 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822370275 0 0) 0x7f747c0a28f0 con 0x7f747c0a53a0 2024-05-08T15:51:14.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): my global_id is 5635 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f747c0a1660 con 0x7f747c0a53a0 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755743779 0 0) 0x7f747c0a2bd0 con 0x7f747c0a5780 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient(hunting): my global_id is 75936 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f747c0a28f0 con 0x7f747c0a5780 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4108475227 0 0) 0x7f7468003520 con 0x7f747c13e310 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 >> v1:172.21.15.81:6790/0 conn(0x7f747c0a5780 legacy=0x7f747c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 >> v1:172.21.15.81:6789/0 conn(0x7f747c0a53a0 legacy=0x7f747c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 monclient: found mon.noname-a 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c0a6730 con 0x7f747c13e310 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 5 monclient: authenticate success, global_id 94932 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f74740017b0 con 0x7f747c13e310 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: dump: 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.377 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7474001ad0 con 0x7f747c13e310 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 1 -- 172.21.15.7:0/1573970457 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7474004be0 con 0x7f747c13e310 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 4 set_mon_vals no callback set 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f7479ffb640 10 monclient: dump: 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.379 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 10 monclient: shutdown 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 1 -- 172.21.15.7:0/1573970457 >> v1:172.21.15.7:6789/0 conn(0x7f747c13e310 legacy=0x7f747c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 1 -- 172.21.15.7:0/1573970457 shutdown_connections 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 1 -- 172.21.15.7:0/1573970457 >> 172.21.15.7:0/1573970457 conn(0x7f747c0a9fa0 msgr2=0x7f747c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 1 -- 172.21.15.7:0/1573970457 shutdown_connections 2024-05-08T15:51:14.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.364+0000 7f74818cf640 1 -- 172.21.15.7:0/1573970457 wait complete. 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.365+0000 7f74818cf640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.365+0000 7f74818cf640 10 monclient: monmap: 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.381 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.365+0000 7f74818cf640 1 Processor -- start 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.365+0000 7f74818cf640 1 -- start start 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.365+0000 7f74818cf640 10 monclient: init 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 10 monclient: _add_conns ranks=[2,7,6,4,3,8,0,1,5] 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 10 monclient(hunting): picked mon.noname-b con 0x7f747c0a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 10 monclient(hunting): picked mon.noname-i con 0x7f747c0a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 10 monclient(hunting): picked mon.noname-d con 0x7f747c13e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c1cdb30 con 0x7f747c0a53a0 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c134170 con 0x7f747c13e310 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f747c0a1e00 con 0x7f747c0a5780 2024-05-08T15:51:14.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f74818cf640 10 monclient(hunting): authenticate will time out at 1082.481127s 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f747affd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f747c0a53a0 0x7f747c1ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:35428/0 (socket says 172.21.15.7:35428) 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f747affd640 1 -- 172.21.15.7:0/3460653253 learned_addr learned my addr 172.21.15.7:0/3460653253 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 5365605 0 0) 0x7f747c1cdb30 con 0x7f747c0a53a0 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): my global_id is 95051 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f745c003520 con 0x7f747c0a53a0 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3524517005 0 0) 0x7f747c134170 con 0x7f747c13e310 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): my global_id is 75732 2024-05-08T15:51:14.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f747c1cdb30 con 0x7f747c13e310 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2011220278 0 0) 0x7f747c0a1e00 con 0x7f747c0a5780 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 10 monclient(hunting): my global_id is 114244 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.366+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f747c134170 con 0x7f747c0a5780 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3721566536 0 0) 0x7f745c003520 con 0x7f747c0a53a0 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f747c0a1e00 con 0x7f747c0a53a0 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f746c001e10 con 0x7f747c0a53a0 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): dump: 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.384 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 530428129 0 0) 0x7f747c1cdb30 con 0x7f747c13e310 2024-05-08T15:51:14.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f745c003520 con 0x7f747c13e310 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7474001060 con 0x7f747c13e310 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): dump: 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.386 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 743145487 0 0) 0x7f747c134170 con 0x7f747c0a5780 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f747c1cdb30 con 0x7f747c0a5780 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7464001990 con 0x7f747c0a5780 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient(hunting): dump: 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.387 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2960826053 0 0) 0x7f747c0a1e00 con 0x7f747c0a53a0 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.81:6792/0 conn(0x7f747c0a5780 legacy=0x7f747c1cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.7:6792/0 conn(0x7f747c13e310 legacy=0x7f747c1d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 monclient: found mon.b 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747c0a43c0 con 0x7f747c0a53a0 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74818cf640 5 monclient: authenticate success, global_id 95051 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74818cf640 10 monclient: _renew_subs 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74818cf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f747c134f40 con 0x7f747c0a53a0 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74818cf640 10 monclient: _renew_subs 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74818cf640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:14.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f747c1339e0 con 0x7f747c0a53a0 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f746c003c40 con 0x7f747c0a53a0 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f746c004130 con 0x7f747c0a53a0 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f74797fa640 4 set_mon_vals no callback set 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.367+0000 7f7473fff640 10 monclient: dump: 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.390 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f74818cf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f74600065c0 con 0x7f7460002860 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f746c0042d0 con 0x7f747c0a53a0 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(197..197 src has 1..197) ==== 3898+0+0 (unknown 2948488469 0 0) 0x7f746c047180 con 0x7f747c0a53a0 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706205151 0 0) 0x7f74600065c0 con 0x7f7460002860 2024-05-08T15:51:14.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 10 monclient(hunting): my global_id is 75741 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f747c0a1e00 con 0x7f7460002860 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f747b7fe640 10 monclient: get_auth_request con 0x7f745c03a460 auth_method 0 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1285375241 0 0) 0x7f747c0a1e00 con 0x7f7460002860 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f74600065c0 con 0x7f7460002860 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7464003930 con 0x7f7460002860 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.368+0000 7f7473fff640 10 monclient: dump: 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.392 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.369+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3812473358 0 0) 0x7f74600065c0 con 0x7f7460002860 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.369+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f746000a150 con 0x7f7460002860 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.369+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7464003cc0 con 0x7f7460002860 2024-05-08T15:51:14.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.369+0000 7f7473fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.369+0000 7f7473fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.369+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.7:6792/0 conn(0x7f7460002860 legacy=0x7f7460004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.369+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.7:6792/0 conn(0x7f7460002860 legacy=0x7f7460004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.375+0000 7f74818cf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.375+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f7460009e40 con 0x7f7460009100 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 336307643 0 0) 0x7f7460009e40 con 0x7f7460009100 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient(hunting): my global_id is 75750 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74600065c0 con 0x7f7460009100 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1062660551 0 0) 0x7f74600065c0 con 0x7f7460009100 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7460009e40 con 0x7f7460009100 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7464004430 con 0x7f7460009100 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient: dump: 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.396 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 442720121 0 0) 0x7f7460009e40 con 0x7f7460009100 2024-05-08T15:51:14.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f746000c3d0 con 0x7f7460009100 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7863 (unknown 0 0 2185594206) 0x7f7464005950 con 0x7f7460009100 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.7:6792/0 conn(0x7f7460009100 legacy=0x7f7460009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.376+0000 7f7473fff640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.7:6792/0 conn(0x7f7460009100 legacy=0x7f7460009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:14.398 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 266690, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.399 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:14.400 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:14.401 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.402 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.403 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.404 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.405 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.406 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.407 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:14.408 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.409 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.410 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.411 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:14.412 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.413 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.414 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:14.415 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:14.416 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.81:6812/847627118 conn(0x7f745c03a460 legacy=0x7f745c03c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 10 monclient: shutdown 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 >> v1:172.21.15.7:6790/0 conn(0x7f747c0a53a0 legacy=0x7f747c1ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f747b7fe640 1 -- 172.21.15.7:0/3460653253 reap_dead start 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f747b7fe640 1 -- 172.21.15.7:0/3460653253 reap_dead start 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 shutdown_connections 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 >> 172.21.15.7:0/3460653253 conn(0x7f747c0a9fa0 msgr2=0x7f747c0a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 shutdown_connections 2024-05-08T15:51:14.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.377+0000 7f74818cf640 1 -- 172.21.15.7:0/3460653253 wait complete. 2024-05-08T15:51:14.419 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:14.419 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:51:14.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:51:14.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.527+0000 7fbd55b91640 10 monclient: get_monmap_and_config 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.527+0000 7fbd55b91640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.527+0000 7fbd55b91640 10 monclient: monmap: 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:14.528848+0000 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:14.528848+0000 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.524 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.525 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.528+0000 7fbd55b91640 1 Processor -- start 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.528+0000 7fbd55b91640 1 -- start start 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.528+0000 7fbd55b91640 10 monclient: init 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 10 monclient: _add_conns ranks=[6,4,8,0,2,1,7,5,3] 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 10 monclient(hunting): picked mon.noname-d con 0x7fbd50137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 10 monclient(hunting): picked mon.noname-c con 0x7fbd500a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 10 monclient(hunting): picked mon.noname-e con 0x7fbd500a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd500a2c70 con 0x7fbd500a49a0 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd500a2f50 con 0x7fbd50137940 2024-05-08T15:51:14.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd500a1670 con 0x7fbd500a41b0 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 10 monclient(hunting): authenticate will time out at 1082.643789s 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4effd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fbd500a49a0 0x7fbd5013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55484/0 (socket says 172.21.15.7:55484) 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4effd640 1 -- 172.21.15.7:0/1335742261 learned_addr learned my addr 172.21.15.7:0/1335742261 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4ffff640 1 --1- 172.21.15.7:0/1335742261 >> v1:172.21.15.7:6793/0 conn(0x7fbd500a41b0 0x7fbd50140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:50460/0 (socket says 172.21.15.7:50460) 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4f7fe640 1 --1- 172.21.15.7:0/1335742261 >> v1:172.21.15.7:6792/0 conn(0x7fbd50137940 0x7fbd50137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:46954/0 (socket says 172.21.15.7:46954) 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014691825 0 0) 0x7fbd500a1670 con 0x7fbd500a41b0 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): my global_id is 94904 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd38003520 con 0x7fbd500a41b0 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613433786 0 0) 0x7fbd500a2c70 con 0x7fbd500a49a0 2024-05-08T15:51:14.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): my global_id is 114268 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd500a1670 con 0x7fbd500a49a0 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 967786402 0 0) 0x7fbd500a2f50 con 0x7fbd50137940 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient(hunting): my global_id is 75759 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd500a2c70 con 0x7fbd50137940 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3744728743 0 0) 0x7fbd500a1670 con 0x7fbd500a49a0 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 >> v1:172.21.15.7:6793/0 conn(0x7fbd500a41b0 legacy=0x7fbd50140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 >> v1:172.21.15.7:6792/0 conn(0x7fbd50137940 legacy=0x7fbd50137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 monclient: found mon.noname-c 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd500a39f0 con 0x7fbd500a49a0 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbd44001790 con 0x7fbd500a49a0 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 5 monclient: authenticate success, global_id 114268 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd55b91640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.529+0000 7fbd4e7fc640 10 monclient: dump: 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.537 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.538 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbd44001a70 con 0x7fbd500a49a0 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd4e7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd4e7fc640 1 -- 172.21.15.7:0/1335742261 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbd44004be0 con 0x7fbd500a49a0 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 4 set_mon_vals no callback set 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd4e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd4e7fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd4e7fc640 10 monclient: dump: 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.539 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.540 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 10 monclient: shutdown 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 1 -- 172.21.15.7:0/1335742261 >> v1:172.21.15.7:6791/0 conn(0x7fbd500a49a0 legacy=0x7fbd5013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 1 -- 172.21.15.7:0/1335742261 shutdown_connections 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 1 -- 172.21.15.7:0/1335742261 >> 172.21.15.7:0/1335742261 conn(0x7fbd500a9fa0 msgr2=0x7fbd500aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 1 -- 172.21.15.7:0/1335742261 shutdown_connections 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.530+0000 7fbd55b91640 1 -- 172.21.15.7:0/1335742261 wait complete. 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.531+0000 7fbd55b91640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.531+0000 7fbd55b91640 10 monclient: monmap: 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:14.541 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.531+0000 7fbd55b91640 1 Processor -- start 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.531+0000 7fbd55b91640 1 -- start start 2024-05-08T15:51:14.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.531+0000 7fbd55b91640 10 monclient: init 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient: _add_conns ranks=[4,3,5,1,8,6,2,7,0] 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient(hunting): picked mon.noname-c con 0x7fbd500a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient(hunting): picked mon.noname-g con 0x7fbd500a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient(hunting): picked mon.noname-h con 0x7fbd50137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd500a1f30 con 0x7fbd500a49a0 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd500a0f60 con 0x7fbd500a41b0 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd50134760 con 0x7fbd50137940 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient(hunting): authenticate will time out at 1082.646921s 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd4f7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fbd500a41b0 0x7fbd50133940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:55492/0 (socket says 172.21.15.7:55492) 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd4f7fe640 1 -- 172.21.15.7:0/1962698387 learned_addr learned my addr 172.21.15.7:0/1962698387 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1363820425 0 0) 0x7fbd500a0f60 con 0x7fbd500a41b0 2024-05-08T15:51:14.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): my global_id is 114277 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd3c003520 con 0x7fbd500a41b0 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3696396413 0 0) 0x7fbd3c003520 con 0x7fbd500a41b0 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbd500a0f60 con 0x7fbd500a41b0 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbd400018d0 con 0x7fbd500a41b0 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): dump: 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.544 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.545 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755908670 0 0) 0x7fbd500a1f30 con 0x7fbd500a49a0 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): my global_id is 75945 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd3c003520 con 0x7fbd500a49a0 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031826979 0 0) 0x7fbd50134760 con 0x7fbd50137940 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient(hunting): my global_id is 5675 2024-05-08T15:51:14.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd500a1f30 con 0x7fbd50137940 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2335397818 0 0) 0x7fbd500a0f60 con 0x7fbd500a41b0 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.81:6791/0 conn(0x7fbd50137940 legacy=0x7fbd501d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.81:6790/0 conn(0x7fbd500a49a0 legacy=0x7fbd50134050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 monclient: found mon.c 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd50137580 con 0x7fbd500a41b0 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd37fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 5 monclient: authenticate success, global_id 114277 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient: _renew_subs 2024-05-08T15:51:14.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.532+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbd501360d0 con 0x7fbd500a41b0 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd55b91640 10 monclient: _renew_subs 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd55b91640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fbd501d5da0 con 0x7fbd500a41b0 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbd40001d00 con 0x7fbd500a41b0 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd37fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbd40004e90 con 0x7fbd500a41b0 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd37fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd4dffb640 4 set_mon_vals no callback set 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd37fff640 10 monclient: dump: 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.548 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd55b91640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbd300065c0 con 0x7fbd30002860 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.533+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fbd40010210 con 0x7fbd500a41b0 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(197..197 src has 1..197) ==== 3898+0+0 (unknown 2948488469 0 0) 0x7fbd40046f50 con 0x7fbd500a41b0 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3794675553 0 0) 0x7fbd300065c0 con 0x7fbd30002860 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient(hunting): my global_id is 94913 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd500a0f60 con 0x7fbd30002860 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd4ffff640 10 monclient: get_auth_request con 0x7fbd3c039ca0 auth_method 0 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3946105171 0 0) 0x7fbd500a0f60 con 0x7fbd30002860 2024-05-08T15:51:14.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbd300065c0 con 0x7fbd30002860 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbd44004b20 con 0x7fbd30002860 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient: dump: 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.551 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1022836562 0 0) 0x7fbd300065c0 con 0x7fbd30002860 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd3000a150 con 0x7fbd30002860 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fbd44001210 con 0x7fbd30002860 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.7:6793/0 conn(0x7fbd30002860 legacy=0x7fbd30004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.534+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.7:6793/0 conn(0x7fbd30002860 legacy=0x7fbd30004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd55b91640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbd30009e40 con 0x7fbd30009100 2024-05-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4293672231 0 0) 0x7fbd30009e40 con 0x7fbd30009100 2024-05-08T15:51:14.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 10 monclient(hunting): my global_id is 94922 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd300065c0 con 0x7fbd30009100 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4259314527 0 0) 0x7fbd300065c0 con 0x7fbd30009100 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbd30009e40 con 0x7fbd30009100 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbd44005000 con 0x7fbd30009100 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.541+0000 7fbd37fff640 10 monclient: dump: 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.555 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3382831234 0 0) 0x7fbd30009e40 con 0x7fbd30009100 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbd3000c3d0 con 0x7fbd30009100 2024-05-08T15:51:14.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1598078639) 0x7fbd440072c0 con 0x7fbd30009100 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd37fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd37fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.7:6793/0 conn(0x7fbd30009100 legacy=0x7fbd30009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd37fff640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.7:6793/0 conn(0x7fbd30009100 legacy=0x7fbd30009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:14.557 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 125094, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.558 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:14.559 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.560 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:14.561 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.562 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:14.563 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.564 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.565 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.566 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.567 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.568 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:14.569 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.570 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.571 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.572 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.573 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.574 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.575 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.576 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:14.577 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.81:6812/847627118 conn(0x7fbd3c039ca0 legacy=0x7fbd3c03c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 10 monclient: shutdown 2024-05-08T15:51:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 >> v1:172.21.15.7:6791/0 conn(0x7fbd500a41b0 legacy=0x7fbd50133940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd4ffff640 1 -- 172.21.15.7:0/1962698387 reap_dead start 2024-05-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd4ffff640 1 -- 172.21.15.7:0/1962698387 reap_dead start 2024-05-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 shutdown_connections 2024-05-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 >> 172.21.15.7:0/1962698387 conn(0x7fbd500a9fa0 msgr2=0x7fbd501307f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 shutdown_connections 2024-05-08T15:51:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.542+0000 7fbd55b91640 1 -- 172.21.15.7:0/1962698387 wait complete. 2024-05-08T15:51:14.579 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:14.579 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:51:14.580 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.687+0000 7f08c5ee6640 10 monclient: get_monmap_and_config 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.687+0000 7f08c5ee6640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.687+0000 7f08c5ee6640 10 monclient: monmap: 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:14.688079+0000 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:14.688079+0000 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.683 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.684 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.684 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.687+0000 7f08c5ee6640 1 Processor -- start 2024-05-08T15:51:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.687+0000 7f08c5ee6640 1 -- start start 2024-05-08T15:51:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.687+0000 7f08c5ee6640 10 monclient: init 2024-05-08T15:51:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 10 monclient: _add_conns ranks=[7,6,5,8,1,4,0,2,3] 2024-05-08T15:51:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 10 monclient(hunting): picked mon.noname-i con 0x7f08c00a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:14.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 10 monclient(hunting): picked mon.noname-d con 0x7f08c00a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 10 monclient(hunting): picked mon.noname-h con 0x7f08c013e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c00a1660 con 0x7f08c013e310 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c00a28f0 con 0x7f08c00a5780 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c00a2bd0 con 0x7f08c00a53a0 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08c5ee6640 10 monclient(hunting): authenticate will time out at 1082.803059s 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08beffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f08c00a5780 0x7f08c013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:40978/0 (socket says 172.21.15.7:40978) 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08beffd640 1 -- 172.21.15.7:0/1529320869 learned_addr learned my addr 172.21.15.7:0/1529320869 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3395334638 0 0) 0x7f08c00a28f0 con 0x7f08c00a5780 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08be7fc640 10 monclient(hunting): my global_id is 75768 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.688+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08a8003520 con 0x7f08c00a5780 2024-05-08T15:51:14.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678613115 0 0) 0x7f08c00a1660 con 0x7f08c013e310 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): my global_id is 5684 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c00a28f0 con 0x7f08c013e310 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1457738042 0 0) 0x7f08c00a2bd0 con 0x7f08c00a53a0 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient(hunting): my global_id is 114253 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c00a1660 con 0x7f08c00a53a0 2024-05-08T15:51:14.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2814458352 0 0) 0x7f08a8003520 con 0x7f08c00a5780 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 >> v1:172.21.15.81:6792/0 conn(0x7f08c00a53a0 legacy=0x7f08c00a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 >> v1:172.21.15.81:6791/0 conn(0x7f08c013e310 legacy=0x7f08c0140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 monclient: found mon.noname-d 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c00a6730 con 0x7f08c00a5780 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b40017b0 con 0x7f08c00a5780 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: dump: 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.696 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.697 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.698 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 5 monclient: authenticate success, global_id 75768 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f08b4001a10 con 0x7f08c00a5780 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 1 -- 172.21.15.7:0/1529320869 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b4004be0 con 0x7f08c00a5780 2024-05-08T15:51:14.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 4 set_mon_vals no callback set 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08be7fc640 10 monclient: dump: 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.700 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 10 monclient: shutdown 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 1 -- 172.21.15.7:0/1529320869 >> v1:172.21.15.7:6792/0 conn(0x7f08c00a5780 legacy=0x7f08c013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 1 -- 172.21.15.7:0/1529320869 shutdown_connections 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 1 -- 172.21.15.7:0/1529320869 >> 172.21.15.7:0/1529320869 conn(0x7f08c00a9fa0 msgr2=0x7f08c00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 1 -- 172.21.15.7:0/1529320869 shutdown_connections 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.689+0000 7f08c5ee6640 1 -- 172.21.15.7:0/1529320869 wait complete. 2024-05-08T15:51:14.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.690+0000 7f08c5ee6640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.690+0000 7f08c5ee6640 10 monclient: monmap: 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.702 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.690+0000 7f08c5ee6640 1 Processor -- start 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.690+0000 7f08c5ee6640 1 -- start start 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.690+0000 7f08c5ee6640 10 monclient: init 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 10 monclient: _add_conns ranks=[2,4,7,0,8,6,1,3,5] 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 10 monclient(hunting): picked mon.noname-b con 0x7f08c00a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 10 monclient(hunting): picked mon.noname-c con 0x7f08c00a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 10 monclient(hunting): picked mon.noname-i con 0x7f08c013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c0138310 con 0x7f08c00a53a0 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c0137220 con 0x7f08c00a5780 2024-05-08T15:51:14.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c0131510 con 0x7f08c013e310 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08bf7fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f08c00a53a0 0x7f08c01306f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:56372/0 (socket says 172.21.15.7:56372) 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08bf7fe640 1 -- 172.21.15.7:0/868184656 learned_addr learned my addr 172.21.15.7:0/868184656 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08c5ee6640 10 monclient(hunting): authenticate will time out at 1082.806207s 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2699115377 0 0) 0x7f08c0137220 con 0x7f08c00a5780 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): my global_id is 114286 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08ac003520 con 0x7f08c00a5780 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2494181382 0 0) 0x7f08c0138310 con 0x7f08c00a53a0 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): my global_id is 95060 2024-05-08T15:51:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c0137220 con 0x7f08c00a53a0 2024-05-08T15:51:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 548239702 0 0) 0x7f08c0131510 con 0x7f08c013e310 2024-05-08T15:51:14.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 10 monclient(hunting): my global_id is 114262 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.691+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c0138310 con 0x7f08c013e310 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3279208554 0 0) 0x7f08ac003520 con 0x7f08c00a5780 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08c0131510 con 0x7f08c00a5780 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b4004b20 con 0x7f08c00a5780 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): dump: 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.706 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3371261699 0 0) 0x7f08c0137220 con 0x7f08c00a53a0 2024-05-08T15:51:14.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08ac003520 con 0x7f08c00a53a0 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b0001a10 con 0x7f08c00a53a0 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): dump: 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.708 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.709 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2856359403 0 0) 0x7f08c0138310 con 0x7f08c013e310 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08c0137220 con 0x7f08c013e310 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b8001cf0 con 0x7f08c013e310 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient(hunting): dump: 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.711 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4006664731 0 0) 0x7f08c0131510 con 0x7f08c00a5780 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.81:6792/0 conn(0x7f08c013e310 legacy=0x7f08c01d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.7:6790/0 conn(0x7f08c00a53a0 legacy=0x7f08c01306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 monclient: found mon.c 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c00a1e00 con 0x7f08c00a5780 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08c5ee6640 5 monclient: authenticate success, global_id 114286 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08c5ee6640 10 monclient: _renew_subs 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08c5ee6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f08c00a43c0 con 0x7f08c00a5780 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08c5ee6640 10 monclient: _renew_subs 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08c5ee6640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f08c0136f40 con 0x7f08c00a5780 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f08b4001710 con 0x7f08c00a5780 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08bdffb640 4 set_mon_vals no callback set 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b4005220 con 0x7f08c00a5780 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.692+0000 7f08a7fff640 10 monclient: dump: 2024-05-08T15:51:14.713 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.714 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08c5ee6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f08a00065c0 con 0x7f08a0002860 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f08b40105a0 con 0x7f08c00a5780 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(197..197 src has 1..197) ==== 3898+0+0 (unknown 2948488469 0 0) 0x7f08b4046df0 con 0x7f08c00a5780 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404190772 0 0) 0x7f08a00065c0 con 0x7f08a0002860 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 10 monclient(hunting): my global_id is 5644 2024-05-08T15:51:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c0131510 con 0x7f08a0002860 2024-05-08T15:51:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08bffff640 10 monclient: get_auth_request con 0x7f08ac03a3e0 auth_method 0 2024-05-08T15:51:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2508874289 0 0) 0x7f08c0131510 con 0x7f08a0002860 2024-05-08T15:51:14.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08a00065c0 con 0x7f08a0002860 2024-05-08T15:51:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b0003bc0 con 0x7f08a0002860 2024-05-08T15:51:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:51:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.693+0000 7f08a7fff640 10 monclient: dump: 2024-05-08T15:51:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.717 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.694+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4249777720 0 0) 0x7f08a00065c0 con 0x7f08a0002860 2024-05-08T15:51:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.694+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08a000a150 con 0x7f08a0002860 2024-05-08T15:51:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.694+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f08b0003e30 con 0x7f08a0002860 2024-05-08T15:51:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.694+0000 7f08a7fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.694+0000 7f08a7fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.694+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.81:6789/0 conn(0x7f08a0002860 legacy=0x7f08a0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.694+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.81:6789/0 conn(0x7f08a0002860 legacy=0x7f08a0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08c5ee6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f08a0009e20 con 0x7f08a00090e0 2024-05-08T15:51:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3272241022 0 0) 0x7f08a0009e20 con 0x7f08a00090e0 2024-05-08T15:51:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 10 monclient(hunting): my global_id is 5653 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08a00065c0 con 0x7f08a00090e0 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1138617779 0 0) 0x7f08a00065c0 con 0x7f08a00090e0 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f08a0009e20 con 0x7f08a00090e0 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f08b00057c0 con 0x7f08a00090e0 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 10 monclient: dump: 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.723 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1988257737 0 0) 0x7f08a0009e20 con 0x7f08a00090e0 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.701+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f08a000c390 con 0x7f08a00090e0 2024-05-08T15:51:14.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7695 (unknown 0 0 1827681481) 0x7f08b00058f0 con 0x7f08a00090e0 2024-05-08T15:51:14.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08a7fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:14.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08a7fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:14.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.81:6789/0 conn(0x7f08a00090e0 legacy=0x7f08a0009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08a7fff640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.81:6789/0 conn(0x7f08a00090e0 legacy=0x7f08a0009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.725 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:14.725 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 267018, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:14.726 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.727 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.728 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:14.729 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.730 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.731 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.732 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.733 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.734 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.735 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.736 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.737 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.738 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.739 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.740 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.741 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.742 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.743 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.744 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:14.744 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.81:6812/847627118 conn(0x7f08ac03a3e0 legacy=0x7f08ac03c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 10 monclient: shutdown 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 >> v1:172.21.15.7:6791/0 conn(0x7f08c00a5780 legacy=0x7f08c0130e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08bffff640 1 -- 172.21.15.7:0/868184656 reap_dead start 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08bffff640 1 -- 172.21.15.7:0/868184656 reap_dead start 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 shutdown_connections 2024-05-08T15:51:14.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 >> 172.21.15.7:0/868184656 conn(0x7f08c00a9fa0 msgr2=0x7f08c00a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 shutdown_connections 2024-05-08T15:51:14.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.702+0000 7f08c5ee6640 1 -- 172.21.15.7:0/868184656 wait complete. 2024-05-08T15:51:14.746 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:14.746 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:51:14.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:51:14.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.853+0000 7fd0b2338640 10 monclient: get_monmap_and_config 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.853+0000 7fd0b2338640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.853+0000 7fd0b2338640 10 monclient: monmap: 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:14.854809+0000 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:14.854809+0000 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.850 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.851 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.851 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.854+0000 7fd0b2338640 1 Processor -- start 2024-05-08T15:51:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.854+0000 7fd0b2338640 1 -- start start 2024-05-08T15:51:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.854+0000 7fd0b2338640 10 monclient: init 2024-05-08T15:51:14.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 10 monclient: _add_conns ranks=[3,4,2,0,6,1,5,8,7] 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 10 monclient(hunting): picked mon.noname-g con 0x7fd0ac137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 10 monclient(hunting): picked mon.noname-c con 0x7fd0ac0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 10 monclient(hunting): picked mon.noname-b con 0x7fd0ac0a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ac0a2c70 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ac0a2f50 con 0x7fd0ac137940 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ac0a1670 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b2338640 10 monclient(hunting): authenticate will time out at 1082.969949s 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b08d5640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd0ac0a41b0 0x7fd0ac140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:56386/0 (socket says 172.21.15.7:56386) 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0b08d5640 1 -- 172.21.15.7:0/1868227465 learned_addr learned my addr 172.21.15.7:0/1868227465 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0a3fff640 1 --1- 172.21.15.7:0/1868227465 >> v1:172.21.15.7:6791/0 conn(0x7fd0ac0a49a0 0x7fd0ac13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:39102/0 (socket says 172.21.15.7:39102) 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3246785462 0 0) 0x7fd0ac0a1670 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): my global_id is 114295 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd098003520 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1512281048 0 0) 0x7fd0ac0a2c70 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): my global_id is 95069 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0ac0a1670 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2872017485 0 0) 0x7fd0ac0a2f50 con 0x7fd0ac137940 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 10 monclient(hunting): my global_id is 75954 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0ac0a2c70 con 0x7fd0ac137940 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4052618936 0 0) 0x7fd098003520 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.855+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 >> v1:172.21.15.81:6790/0 conn(0x7fd0ac137940 legacy=0x7fd0ac137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 >> v1:172.21.15.7:6790/0 conn(0x7fd0ac0a41b0 legacy=0x7fd0ac140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 1 monclient: found mon.noname-c 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ac0a39f0 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd0940017b0 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 5 monclient: authenticate success, global_id 114295 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: dump: 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.865 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd094001a10 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 1 -- 172.21.15.7:0/1868227465 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd094004b20 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 4 set_mon_vals no callback set 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0ab7fe640 10 monclient: dump: 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.867 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 10 monclient: shutdown 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 1 -- 172.21.15.7:0/1868227465 >> v1:172.21.15.7:6791/0 conn(0x7fd0ac0a49a0 legacy=0x7fd0ac13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 1 -- 172.21.15.7:0/1868227465 shutdown_connections 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 1 -- 172.21.15.7:0/1868227465 >> 172.21.15.7:0/1868227465 conn(0x7fd0ac0a9fa0 msgr2=0x7fd0ac0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 1 -- 172.21.15.7:0/1868227465 shutdown_connections 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.856+0000 7fd0b2338640 1 -- 172.21.15.7:0/1868227465 wait complete. 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.857+0000 7fd0b2338640 10 monclient: build_initial_monmap 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.857+0000 7fd0b2338640 10 monclient: monmap: 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:14.868 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.857+0000 7fd0b2338640 1 Processor -- start 2024-05-08T15:51:14.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.857+0000 7fd0b2338640 1 -- start start 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.857+0000 7fd0b2338640 10 monclient: init 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 10 monclient: _add_conns ranks=[2,4,0,1,7,8,3,6,5] 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 10 monclient(hunting): picked mon.noname-b con 0x7fd0ac0a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 10 monclient(hunting): picked mon.noname-c con 0x7fd0ac0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 10 monclient(hunting): picked mon.noname-a con 0x7fd0ac137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ac133820 con 0x7fd0ac137940 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ac135ac0 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0abfff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd0ac0a41b0 0x7fd0ac136ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:56394/0 (socket says 172.21.15.7:56394) 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a3fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fd0ac0a49a0 0x7fd0ac1cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:39104/0 (socket says 172.21.15.7:39104) 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0abfff640 1 -- 172.21.15.7:0/2391705198 learned_addr learned my addr 172.21.15.7:0/2391705198 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ac1357e0 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:14.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0b2338640 10 monclient(hunting): authenticate will time out at 1082.973063s 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3065796875 0 0) 0x7fd0ac1357e0 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): my global_id is 114304 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd08c003520 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3448650849 0 0) 0x7fd0ac135ac0 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): my global_id is 95078 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0ac1357e0 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374346233 0 0) 0x7fd0ac133820 con 0x7fd0ac137940 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): my global_id is 94941 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0ac135ac0 con 0x7fd0ac137940 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1450482252 0 0) 0x7fd0ac135ac0 con 0x7fd0ac137940 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd0ac133820 con 0x7fd0ac137940 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd0a40020c0 con 0x7fd0ac137940 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.858+0000 7fd0a97fa640 10 monclient(hunting): dump: 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.872 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2488367994 0 0) 0x7fd0ac1357e0 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd0ac135ac0 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1362704873 0 0) 0x7fd08c003520 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd0ac1357e0 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd09c0019f0 con 0x7fd0ac0a41b0 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient(hunting): dump: 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.874 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd0940016b0 con 0x7fd0ac0a49a0 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient(hunting): dump: 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.875 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 78452509 0 0) 0x7fd0ac133820 con 0x7fd0ac137940 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.7:6791/0 conn(0x7fd0ac0a49a0 legacy=0x7fd0ac1cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.7:6790/0 conn(0x7fd0ac0a41b0 legacy=0x7fd0ac136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: _finish_hunting 0 2024-05-08T15:51:14.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 monclient: found mon.a 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ac137700 con 0x7fd0ac137940 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: _finish_auth 0 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: _check_auth_tickets 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0b2338640 5 monclient: authenticate success, global_id 94941 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0b2338640 10 monclient: _renew_subs 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0b2338640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ac1cc9d0 con 0x7fd0ac137940 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0b2338640 10 monclient: _renew_subs 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0b2338640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd0ac1d5d20 con 0x7fd0ac137940 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd0a4001950 con 0x7fd0ac137940 2024-05-08T15:51:14.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd0a4005150 con 0x7fd0ac137940 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0aaffd640 4 set_mon_vals no callback set 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.859+0000 7fd0a97fa640 10 monclient: dump: 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.878 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0b2338640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd0900065c0 con 0x7fd090002860 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd0a40104d0 con 0x7fd0ac137940 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(197..197 src has 1..197) ==== 3898+0+0 (unknown 2948488469 0 0) 0x7fd0a4047140 con 0x7fd0ac137940 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a3fff640 10 monclient: get_auth_request con 0x7fd08c03a3d0 auth_method 0 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374791607 0 0) 0x7fd0900065c0 con 0x7fd090002860 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 10 monclient(hunting): my global_id is 75963 2024-05-08T15:51:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0ac133820 con 0x7fd090002860 2024-05-08T15:51:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 930670094 0 0) 0x7fd0ac133820 con 0x7fd090002860 2024-05-08T15:51:14.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd0900065c0 con 0x7fd090002860 2024-05-08T15:51:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd09c003bc0 con 0x7fd090002860 2024-05-08T15:51:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.860+0000 7fd0a97fa640 10 monclient: dump: 2024-05-08T15:51:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.881 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.882 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.861+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3019053435 0 0) 0x7fd0900065c0 con 0x7fd090002860 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.861+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd09000a150 con 0x7fd090002860 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.861+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fd09c003df0 con 0x7fd090002860 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.861+0000 7fd0a97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.861+0000 7fd0a97fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.861+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.81:6790/0 conn(0x7fd090002860 legacy=0x7fd090004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.861+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.81:6790/0 conn(0x7fd090002860 legacy=0x7fd090004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.867+0000 7fd0b2338640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.867+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd090009e20 con 0x7fd0900090e0 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3618594403 0 0) 0x7fd090009e20 con 0x7fd0900090e0 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 10 monclient(hunting): my global_id is 75972 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd0900065c0 con 0x7fd0900090e0 2024-05-08T15:51:14.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2726974343 0 0) 0x7fd0900065c0 con 0x7fd0900090e0 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.868+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd090009e20 con 0x7fd0900090e0 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd09c0057c0 con 0x7fd0900090e0 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 10 monclient: dump: 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:14.886 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1029244974 0 0) 0x7fd090009e20 con 0x7fd0900090e0 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd09000c390 con 0x7fd0900090e0 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 3874483555) 0x7fd09c007af0 con 0x7fd0900090e0 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.81:6790/0 conn(0x7fd0900090e0 legacy=0x7fd090009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0a97fa640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.81:6790/0 conn(0x7fd0900090e0 legacy=0x7fd090009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 267159, 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:14.888 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.889 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.890 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.891 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:14.892 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.893 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.894 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.895 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.896 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.897 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.898 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.899 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.900 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.901 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.902 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.903 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:14.904 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.905 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.906 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 5 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:14.907 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.81:6812/847627118 conn(0x7fd08c03a3d0 legacy=0x7fd08c03c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b2338640 10 monclient: shutdown 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b2338640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 >> v1:172.21.15.7:6789/0 conn(0x7fd0ac137940 legacy=0x7fd0ac1d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b08d5640 1 -- 172.21.15.7:0/2391705198 reap_dead start 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b08d5640 1 -- 172.21.15.7:0/2391705198 reap_dead start 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 shutdown_connections 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 >> 172.21.15.7:0/2391705198 conn(0x7fd0ac0a9fa0 msgr2=0x7fd0ac130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.869+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 shutdown_connections 2024-05-08T15:51:14.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:14.870+0000 7fd0b2338640 1 -- 172.21.15.7:0/2391705198 wait complete. 2024-05-08T15:51:14.910 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:14.910 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:51:14.910 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:51:15.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.017+0000 7f4d5646a640 10 monclient: get_monmap_and_config 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.017+0000 7f4d5646a640 10 monclient: build_initial_monmap 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.017+0000 7f4d5646a640 10 monclient: monmap: 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:15.018858+0000 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:15.018858+0000 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:15.014 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:15.015 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.018+0000 7f4d5646a640 1 Processor -- start 2024-05-08T15:51:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.018+0000 7f4d5646a640 1 -- start start 2024-05-08T15:51:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.018+0000 7f4d5646a640 10 monclient: init 2024-05-08T15:51:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:15.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 10 monclient: _add_conns ranks=[5,8,4,3,1,2,0,7,6] 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 10 monclient(hunting): picked mon.noname-h con 0x7f4d50137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 10 monclient(hunting): picked mon.noname-e con 0x7f4d500a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 10 monclient(hunting): picked mon.noname-c con 0x7f4d500a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d500a2c70 con 0x7f4d500a41b0 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d500a2f50 con 0x7f4d50137940 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d500a1670 con 0x7f4d500a49a0 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 10 monclient(hunting): authenticate will time out at 1083.133925s 2024-05-08T15:51:15.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d54a07640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f4d500a41b0 0x7f4d50140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:39110/0 (socket says 172.21.15.7:39110) 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d54a07640 1 -- 172.21.15.7:0/1357807897 learned_addr learned my addr 172.21.15.7:0/1357807897 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4f7fe640 1 --1- 172.21.15.7:0/1357807897 >> v1:172.21.15.7:6793/0 conn(0x7f4d500a49a0 0x7f4d5013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:48678/0 (socket says 172.21.15.7:48678) 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388697548 0 0) 0x7f4d500a2c70 con 0x7f4d500a41b0 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): my global_id is 114313 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d38003520 con 0x7f4d500a41b0 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2009965238 0 0) 0x7f4d500a1670 con 0x7f4d500a49a0 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): my global_id is 94931 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d500a2c70 con 0x7f4d500a49a0 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114051072 0 0) 0x7f4d500a2f50 con 0x7f4d50137940 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient(hunting): my global_id is 5693 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d500a1670 con 0x7f4d50137940 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 385723837 0 0) 0x7f4d38003520 con 0x7f4d500a41b0 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 >> v1:172.21.15.7:6793/0 conn(0x7f4d500a49a0 legacy=0x7f4d5013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 >> v1:172.21.15.81:6791/0 conn(0x7f4d50137940 legacy=0x7f4d50137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 monclient: found mon.noname-c 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d500a39f0 con 0x7f4d500a41b0 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient: _finish_auth 0 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4d48001b30 con 0x7f4d500a41b0 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 5 monclient: authenticate success, global_id 114313 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d5646a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.019+0000 7f4d4effd640 10 monclient: dump: 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.027 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.028 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d48001e50 con 0x7f4d500a41b0 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d4effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d4effd640 1 -- 172.21.15.7:0/1357807897 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4d480020c0 con 0x7f4d500a41b0 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 4 set_mon_vals no callback set 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d4effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d4effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d4effd640 10 monclient: dump: 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.029 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 10 monclient: shutdown 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:15.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 1 -- 172.21.15.7:0/1357807897 >> v1:172.21.15.7:6791/0 conn(0x7f4d500a41b0 legacy=0x7f4d50140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 1 -- 172.21.15.7:0/1357807897 shutdown_connections 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 1 -- 172.21.15.7:0/1357807897 >> 172.21.15.7:0/1357807897 conn(0x7f4d500a9fa0 msgr2=0x7f4d500aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 1 -- 172.21.15.7:0/1357807897 shutdown_connections 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.020+0000 7f4d5646a640 1 -- 172.21.15.7:0/1357807897 wait complete. 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.021+0000 7f4d5646a640 10 monclient: build_initial_monmap 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.021+0000 7f4d5646a640 10 monclient: monmap: 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:15.031 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.021+0000 7f4d5646a640 1 Processor -- start 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.021+0000 7f4d5646a640 1 -- start start 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.021+0000 7f4d5646a640 10 monclient: init 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 10 monclient: _add_conns ranks=[3,5,0,7,2,1,8,6,4] 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 10 monclient(hunting): picked mon.noname-g con 0x7f4d500a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 10 monclient(hunting): picked mon.noname-h con 0x7f4d500a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 10 monclient(hunting): picked mon.noname-a con 0x7f4d50137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:15.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d50135b90 con 0x7f4d50137940 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d500a1f30 con 0x7f4d500a41b0 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d500a0f60 con 0x7f4d500a49a0 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d5646a640 10 monclient(hunting): authenticate will time out at 1083.136982s 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d54a07640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f4d50137940 0x7f4d501d4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:42510/0 (socket says 172.21.15.7:42510) 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d54a07640 1 -- 172.21.15.7:0/4252951456 learned_addr learned my addr 172.21.15.7:0/4252951456 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2668726641 0 0) 0x7f4d50135b90 con 0x7f4d50137940 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): my global_id is 94950 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d3c003520 con 0x7f4d50137940 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3014425851 0 0) 0x7f4d3c003520 con 0x7f4d50137940 2024-05-08T15:51:15.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d50135b90 con 0x7f4d50137940 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4d48001af0 con 0x7f4d50137940 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): dump: 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.034 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2783094706 0 0) 0x7f4d500a0f60 con 0x7f4d500a49a0 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): my global_id is 5702 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d3c003520 con 0x7f4d500a49a0 2024-05-08T15:51:15.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 443924356 0 0) 0x7f4d500a1f30 con 0x7f4d500a41b0 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 10 monclient(hunting): my global_id is 75981 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.022+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d500a0f60 con 0x7f4d500a41b0 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3252611135 0 0) 0x7f4d50135b90 con 0x7f4d50137940 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.81:6791/0 conn(0x7f4d500a49a0 legacy=0x7f4d501d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.81:6790/0 conn(0x7f4d500a41b0 legacy=0x7f4d50135fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:15.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 1 monclient: found mon.a 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d501338a0 con 0x7f4d50137940 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: _finish_auth 0 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 5 monclient: authenticate success, global_id 94950 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 10 monclient: _renew_subs 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f4d501358b0 con 0x7f4d50137940 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 10 monclient: _renew_subs 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:15.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f4d501d5a90 con 0x7f4d50137940 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f4d48002060 con 0x7f4d50137940 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4d480056c0 con 0x7f4d50137940 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4cff9640 10 monclient: dump: 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.038 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d4e7fc640 4 set_mon_vals no callback set 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.023+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4d501352f0 con 0x7f4d5009ba50 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f4d48010a40 con 0x7f4d50137940 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(197..197 src has 1..197) ==== 3898+0+0 (unknown 2948488469 0 0) 0x7f4d480472a0 con 0x7f4d50137940 2024-05-08T15:51:15.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2190866456 0 0) 0x7f4d501352f0 con 0x7f4d5009ba50 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 10 monclient(hunting): my global_id is 5711 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d50135b90 con 0x7f4d5009ba50 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4f7fe640 10 monclient: get_auth_request con 0x7f4d3c039d20 auth_method 0 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2713040075 0 0) 0x7f4d50135b90 con 0x7f4d5009ba50 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d501352f0 con 0x7f4d5009ba50 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4d40001900 con 0x7f4d5009ba50 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 10 monclient: dump: 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.040 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4044814053 0 0) 0x7f4d501352f0 con 0x7f4d5009ba50 2024-05-08T15:51:15.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.024+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d500008d0 con 0x7f4d5009ba50 2024-05-08T15:51:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.025+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f4d40001cd0 con 0x7f4d5009ba50 2024-05-08T15:51:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.025+0000 7f4d4cff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.025+0000 7f4d4cff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.025+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.81:6791/0 conn(0x7f4d5009ba50 legacy=0x7f4d5009be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.025+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.81:6791/0 conn(0x7f4d5009ba50 legacy=0x7f4d5009be30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.031+0000 7f4d5646a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.031+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f4d5009c920 con 0x7f4d5009c540 2024-05-08T15:51:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2047487944 0 0) 0x7f4d5009c920 con 0x7f4d5009c540 2024-05-08T15:51:15.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 10 monclient(hunting): my global_id is 5720 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d501352f0 con 0x7f4d5009c540 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 555774885 0 0) 0x7f4d501352f0 con 0x7f4d5009c540 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f4d5009c920 con 0x7f4d5009c540 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f4d40003c70 con 0x7f4d5009c540 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 10 monclient: dump: 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.044 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.045 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.046 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.046 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1302912688 0 0) 0x7f4d5009c920 con 0x7f4d5009c540 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.032+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4d500753d0 con 0x7f4d5009c540 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 882495121) 0x7f4d40006020 con 0x7f4d5009c540 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d4cff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d4cff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.81:6791/0 conn(0x7f4d5009c540 legacy=0x7f4d501d9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d4cff9640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.81:6791/0 conn(0x7f4d5009c540 legacy=0x7f4d501d9c80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:15.047 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 267338, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:15.048 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:15.049 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:15.050 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.051 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:15.052 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:15.053 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.054 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.055 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.057 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:15.058 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.059 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:15.060 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.061 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:15.062 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.063 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:15.064 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.065 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:15.066 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:15.067 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.81:6812/847627118 conn(0x7f4d3c039d20 legacy=0x7f4d3c03c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 10 monclient: shutdown 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 >> v1:172.21.15.7:6789/0 conn(0x7f4d50137940 legacy=0x7f4d501d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d54a07640 1 -- 172.21.15.7:0/4252951456 reap_dead start 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d54a07640 1 -- 172.21.15.7:0/4252951456 reap_dead start 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 shutdown_connections 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 >> 172.21.15.7:0/4252951456 conn(0x7f4d500a9fa0 msgr2=0x7f4d50130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 shutdown_connections 2024-05-08T15:51:15.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.033+0000 7f4d5646a640 1 -- 172.21.15.7:0/4252951456 wait complete. 2024-05-08T15:51:15.070 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:15.070 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:51:15.070 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:51:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.179+0000 7fed24edf640 10 monclient: get_monmap_and_config 2024-05-08T15:51:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.179+0000 7fed24edf640 10 monclient: build_initial_monmap 2024-05-08T15:51:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.179+0000 7fed24edf640 10 monclient: monmap: 2024-05-08T15:51:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:15.180164+0000 2024-05-08T15:51:15.175 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:15.180164+0000 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:15.176 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.179+0000 7fed24edf640 1 Processor -- start 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.179+0000 7fed24edf640 1 -- start start 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.179+0000 7fed24edf640 10 monclient: init 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 10 monclient: _add_conns ranks=[6,1,7,3,8,4,0,2,5] 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 10 monclient(hunting): picked mon.noname-d con 0x7fed2006e520 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 10 monclient(hunting): picked mon.noname-f con 0x7fed2006f010 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 10 monclient(hunting): picked mon.noname-i con 0x7fed2013aa90 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed20070be0 con 0x7fed2006f010 2024-05-08T15:51:15.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed200711a0 con 0x7fed2006e520 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2006fe50 con 0x7fed2013aa90 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed24edf640 10 monclient(hunting): authenticate will time out at 1083.295167s 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1e59c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fed2006e520 0x7fed2006e900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:40990/0 (socket says 172.21.15.7:40990) 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1e59c640 1 -- 172.21.15.7:0/350098289 learned_addr learned my addr 172.21.15.7:0/350098289 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 868222377 0 0) 0x7fed200711a0 con 0x7fed2006e520 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1d59a640 10 monclient(hunting): my global_id is 75777 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.180+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed08003520 con 0x7fed2006e520 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2562186647 0 0) 0x7fed20070be0 con 0x7fed2006f010 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient(hunting): my global_id is 5662 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed200711a0 con 0x7fed2006f010 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2913685860 0 0) 0x7fed08003520 con 0x7fed2006e520 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 >> v1:172.21.15.81:6792/0 conn(0x7fed2013aa90 legacy=0x7fed2013ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 >> v1:172.21.15.81:6789/0 conn(0x7fed2006f010 legacy=0x7fed20139370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: _finish_hunting 0 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 monclient: found mon.noname-d 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed20072cc0 con 0x7fed2006e520 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: _finish_auth 0 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: _check_auth_tickets 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:15.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed100017b0 con 0x7fed2006e520 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 5 monclient: authenticate success, global_id 75777 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: dump: 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.188 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fed10001a10 con 0x7fed2006e520 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 1 -- 172.21.15.7:0/350098289 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed10004be0 con 0x7fed2006e520 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 4 set_mon_vals no callback set 2024-05-08T15:51:15.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed1d59a640 10 monclient: dump: 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.190 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 10 monclient: shutdown 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 1 -- 172.21.15.7:0/350098289 >> v1:172.21.15.7:6792/0 conn(0x7fed2006e520 legacy=0x7fed2006e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 1 -- 172.21.15.7:0/350098289 shutdown_connections 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 1 -- 172.21.15.7:0/350098289 >> 172.21.15.7:0/350098289 conn(0x7fed200a6280 msgr2=0x7fed200a2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 1 -- 172.21.15.7:0/350098289 shutdown_connections 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.181+0000 7fed24edf640 1 -- 172.21.15.7:0/350098289 wait complete. 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.182+0000 7fed24edf640 10 monclient: build_initial_monmap 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.182+0000 7fed24edf640 10 monclient: monmap: 2024-05-08T15:51:15.191 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:15.192 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.182+0000 7fed24edf640 1 Processor -- start 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.182+0000 7fed24edf640 1 -- start start 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.182+0000 7fed24edf640 10 monclient: init 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 10 monclient: _add_conns ranks=[7,8,5,1,4,0,6,2,3] 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 10 monclient(hunting): picked mon.noname-i con 0x7fed2006e520 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 10 monclient(hunting): picked mon.noname-e con 0x7fed2006f010 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 10 monclient(hunting): picked mon.noname-h con 0x7fed2013aa90 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2011d730 con 0x7fed2013aa90 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2011da10 con 0x7fed2006e520 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed2011dcf0 con 0x7fed2006f010 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed24edf640 10 monclient(hunting): authenticate will time out at 1083.298034s 2024-05-08T15:51:15.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed1dd9b640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fed2006f010 0x7fed201cd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:48680/0 (socket says 172.21.15.7:48680) 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed1dd9b640 1 -- 172.21.15.7:0/4088211091 learned_addr learned my addr 172.21.15.7:0/4088211091 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167636632 0 0) 0x7fed2011dcf0 con 0x7fed2006f010 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): my global_id is 94940 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed00003520 con 0x7fed2006f010 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2400518942 0 0) 0x7fed2011da10 con 0x7fed2006e520 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): my global_id is 114280 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed2011dcf0 con 0x7fed2006e520 2024-05-08T15:51:15.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31614185 0 0) 0x7fed2011d730 con 0x7fed2013aa90 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 10 monclient(hunting): my global_id is 5729 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.183+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed2011da10 con 0x7fed2013aa90 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 950386897 0 0) 0x7fed00003520 con 0x7fed2006f010 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fed2011d730 con 0x7fed2006f010 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed14001ab0 con 0x7fed2006f010 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): dump: 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.195 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.196 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.197 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2865185026 0 0) 0x7fed2011dcf0 con 0x7fed2006e520 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fed00003520 con 0x7fed2006e520 2024-05-08T15:51:15.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed10004b20 con 0x7fed2006e520 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): dump: 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.199 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1753100812 0 0) 0x7fed2011da10 con 0x7fed2013aa90 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fed2011dcf0 con 0x7fed2013aa90 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed18001d50 con 0x7fed2013aa90 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:15.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient(hunting): dump: 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.201 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1978087278 0 0) 0x7fed2011d730 con 0x7fed2006f010 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.81:6792/0 conn(0x7fed2006e520 legacy=0x7fed20133640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.81:6791/0 conn(0x7fed2013aa90 legacy=0x7fed201d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 monclient: found mon.e 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed20133d50 con 0x7fed2006f010 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: _finish_auth 0 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:15.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed24edf640 5 monclient: authenticate success, global_id 94940 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed24edf640 10 monclient: _renew_subs 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed24edf640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fed201c8f00 con 0x7fed2006f010 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed24edf640 10 monclient: _renew_subs 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed24edf640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fed201c80d0 con 0x7fed2006f010 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fed140038c0 con 0x7fed2006f010 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed14003e30 con 0x7fed2006f010 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed1cd99640 4 set_mon_vals no callback set 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.184+0000 7fed0effd640 10 monclient: dump: 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.203 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed24edf640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fed040065c0 con 0x7fed04002860 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fed14004090 con 0x7fed2006f010 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(197..197 src has 1..197) ==== 3898+0+0 (unknown 2948488469 0 0) 0x7fed14046e50 con 0x7fed2006f010 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375020296 0 0) 0x7fed040065c0 con 0x7fed04002860 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 10 monclient(hunting): my global_id is 114289 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed2011d730 con 0x7fed04002860 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed1ed9d640 10 monclient: get_auth_request con 0x7fed0003a470 auth_method 0 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2040910526 0 0) 0x7fed2011d730 con 0x7fed04002860 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fed040065c0 con 0x7fed04002860 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed10001770 con 0x7fed04002860 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.185+0000 7fed0effd640 10 monclient: dump: 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.206 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.186+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3783766488 0 0) 0x7fed040065c0 con 0x7fed04002860 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.186+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed0400a150 con 0x7fed04002860 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.186+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fed10004f30 con 0x7fed04002860 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.186+0000 7fed0effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.186+0000 7fed0effd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.186+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.81:6792/0 conn(0x7fed04002860 legacy=0x7fed04004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.186+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.81:6792/0 conn(0x7fed04002860 legacy=0x7fed04004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.192+0000 7fed24edf640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.192+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fed04009e40 con 0x7fed04009100 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895845870 0 0) 0x7fed04009e40 con 0x7fed04009100 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 10 monclient(hunting): my global_id is 114298 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed040065c0 con 0x7fed04009100 2024-05-08T15:51:15.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1544351579 0 0) 0x7fed040065c0 con 0x7fed04009100 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fed04009e40 con 0x7fed04009100 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fed10005610 con 0x7fed04009100 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 10 monclient: dump: 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:15.210 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3926402222 0 0) 0x7fed04009e40 con 0x7fed04009100 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.193+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fed0400c3d0 con 0x7fed04009100 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7522 (unknown 0 0 203422871) 0x7fed10006a40 con 0x7fed04009100 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed0effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed0effd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.81:6792/0 conn(0x7fed04009100 legacy=0x7fed04009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed0effd640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.81:6792/0 conn(0x7fed04009100 legacy=0x7fed04009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 28, 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 10058, 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:15.212 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:15.213 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:15.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:15.215 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:15.216 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:15.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:15.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.222 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.224 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:15.226 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:15.228 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:15.229 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:15.230 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:15.230 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.81:6812/847627118 conn(0x7fed0003a470 legacy=0x7fed0003c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 10 monclient: shutdown 2024-05-08T15:51:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 >> v1:172.21.15.7:6793/0 conn(0x7fed2006f010 legacy=0x7fed201cd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:15.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed1ed9d640 1 -- 172.21.15.7:0/4088211091 reap_dead start 2024-05-08T15:51:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed1ed9d640 1 -- 172.21.15.7:0/4088211091 reap_dead start 2024-05-08T15:51:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 shutdown_connections 2024-05-08T15:51:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 >> 172.21.15.7:0/4088211091 conn(0x7fed200a6280 msgr2=0x7fed200a4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 shutdown_connections 2024-05-08T15:51:15.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:15.194+0000 7fed24edf640 1 -- 172.21.15.7:0/4088211091 wait complete. 2024-05-08T15:51:15.232 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:51:15.232 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c', 'd'] 2024-05-08T15:51:15.232 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a', 'i'] 2024-05-08T15:51:15.232 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-05-08T15:51:15.233 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T15:51:15.233 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T15:51:15.233 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:51:15.243 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T15:51:15.243 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.d 2024-05-08T15:51:15.243 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-05-08T15:51:15.243 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-05-08T15:51:15.243 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:51:15.295 INFO:tasks.ceph.mon.d:Stopped 2024-05-08T15:51:15.295 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-05-08T15:51:15.297 INFO:tasks.ceph.mon.a:Sent signal 19 2024-05-08T15:51:15.297 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-05-08T15:51:15.298 INFO:tasks.ceph.mon.i:Sent signal 19 2024-05-08T15:51:15.298 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-05-08T15:51:30.298 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-05-08T15:51:30.299 INFO:tasks.ceph.mon.a:Sent signal 18 2024-05-08T15:51:30.299 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-05-08T15:51:30.299 INFO:tasks.ceph.mon.i:Sent signal 18 2024-05-08T15:51:30.299 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 7 2024-05-08T15:51:30.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:51:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.417+0000 7f3f60b60640 10 monclient: get_monmap_and_config 2024-05-08T15:51:30.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.417+0000 7f3f60b60640 10 monclient: build_initial_monmap 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.417+0000 7f3f60b60640 10 monclient: monmap: 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:30.419114+0000 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:30.419114+0000 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:30.415 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:30.416 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:30.416 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:30.416 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:30.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.417+0000 7f3f60b60640 1 Processor -- start 2024-05-08T15:51:30.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.417+0000 7f3f60b60640 1 -- start start 2024-05-08T15:51:30.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.417+0000 7f3f60b60640 10 monclient: init 2024-05-08T15:51:30.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:30.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 10 monclient: _add_conns ranks=[2,6,3,0,7,1,4,5,8] 2024-05-08T15:51:30.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 10 monclient(hunting): picked mon.noname-b con 0x7f3f5c0a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 10 monclient(hunting): picked mon.noname-d con 0x7f3f5c0a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 10 monclient(hunting): picked mon.noname-g con 0x7f3f5c13e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0a1660 con 0x7f3f5c0a53a0 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0a28f0 con 0x7f3f5c13e310 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0a2bd0 con 0x7f3f5c0a5780 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f60b60640 10 monclient(hunting): authenticate will time out at 1098.534154s 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f59d9b640 1 -- >> v1:172.21.15.7:6792/0 conn(0x7f3f5c0a5780 legacy=0x7f3f5c13cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f5a59c640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f3f5c0a53a0 0x7f3f5c0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:45776/0 (socket says 172.21.15.7:45776) 2024-05-08T15:51:30.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.418+0000 7f3f5a59c640 1 -- 172.21.15.7:0/2258950301 learned_addr learned my addr 172.21.15.7:0/2258950301 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:30.619+0000 7f3f59d9b640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.7:6792/0 conn(0x7f3f5c0a5780 legacy=0x7f3f5c13cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:31.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:31.020+0000 7f3f59d9b640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.7:6792/0 conn(0x7f3f5c0a5780 legacy=0x7f3f5c13cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:31.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:31.821+0000 7f3f59d9b640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.7:6792/0 conn(0x7f3f5c0a5780 legacy=0x7f3f5c13cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient(hunting): tick 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 1 monclient(hunting): continuing hunt 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.7:6792/0 conn(0x7f3f5c0a5780 legacy=0x7f3f5c13cbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.81:6790/0 conn(0x7f3f5c13e310 legacy=0x7f3f5c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.7:6790/0 conn(0x7f3f5c0a53a0 legacy=0x7f3f5c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient: _add_conns ranks=[8,0,7,4,5,1] 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient(hunting): picked mon.noname-e con 0x7f3f3c0037c0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient(hunting): picked mon.noname-a con 0x7f3f3c007450 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient(hunting): picked mon.noname-i con 0x7f3f3c00b1c0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 1 -- 172.21.15.7:0/2258950301 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0a6730 con 0x7f3f3c007450 2024-05-08T15:51:33.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 1 -- 172.21.15.7:0/2258950301 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c0a2bd0 con 0x7f3f3c00b1c0 2024-05-08T15:51:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 1 -- 172.21.15.7:0/2258950301 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c011000 con 0x7f3f3c0037c0 2024-05-08T15:51:33.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:33.419+0000 7f3f4bfff640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:34.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.269+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1959764390 0 0) 0x7f3f3c011000 con 0x7f3f3c0037c0 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.269+0000 7f3f5959a640 10 monclient(hunting): _negotiate 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.269+0000 7f3f5959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.269+0000 7f3f5959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.269+0000 7f3f5959a640 10 monclient(hunting): my global_id is 94949 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.269+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f44003520 con 0x7f3f3c0037c0 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1713884174 0 0) 0x7f3f5c0a2bd0 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient(hunting): _negotiate 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient(hunting): my global_id is 114307 2024-05-08T15:51:34.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f3c011000 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1925458275 0 0) 0x7f3f3c011000 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.7:6793/0 conn(0x7f3f3c0037c0 legacy=0x7f3f3c005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.7:6789/0 conn(0x7f3f3c007450 legacy=0x7f3f3c0098f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5ad9d640 1 -- 172.21.15.7:0/2258950301 reap_dead start 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: _finish_hunting 0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 monclient: found mon.noname-i 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.81:6792/0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c012290 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: _finish_auth 0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: _check_auth_tickets 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3f54001ab0 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:34.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: dump: 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:34.270 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f60b60640 5 monclient: authenticate success, global_id 114307 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f60b60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 <== mon.7 v1:172.21.15.81:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3f54001db0 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 1 -- 172.21.15.7:0/2258950301 <== mon.7 v1:172.21.15.81:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3f54003cd0 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f60b60640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f60b60640 4 set_mon_vals no callback set 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.270+0000 7f3f5959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.271+0000 7f3f5959a640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.271+0000 7f3f5959a640 10 monclient: dump: 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:34.271 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:34.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.271+0000 7f3f60b60640 10 monclient: shutdown 2024-05-08T15:51:34.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.271+0000 7f3f60b60640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:34.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.271+0000 7f3f60b60640 1 -- 172.21.15.7:0/2258950301 >> v1:172.21.15.81:6792/0 conn(0x7f3f3c00b1c0 legacy=0x7f3f3c00d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:34.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.272+0000 7f3f60b60640 1 -- 172.21.15.7:0/2258950301 shutdown_connections 2024-05-08T15:51:34.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.272+0000 7f3f60b60640 1 -- 172.21.15.7:0/2258950301 >> 172.21.15.7:0/2258950301 conn(0x7f3f5c0a9fa0 msgr2=0x7f3f5c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:34.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.272+0000 7f3f60b60640 1 -- 172.21.15.7:0/2258950301 shutdown_connections 2024-05-08T15:51:34.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.272+0000 7f3f60b60640 1 -- 172.21.15.7:0/2258950301 wait complete. 2024-05-08T15:51:34.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.273+0000 7f3f60b60640 10 monclient: build_initial_monmap 2024-05-08T15:51:34.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.273+0000 7f3f60b60640 10 monclient: monmap: 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:34.278 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.273+0000 7f3f60b60640 1 Processor -- start 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.273+0000 7f3f60b60640 1 -- start start 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.273+0000 7f3f60b60640 10 monclient: init 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:34.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 10 monclient: _add_conns ranks=[6,0,8,7,4,5,3,1,2] 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 10 monclient(hunting): picked mon.noname-d con 0x7f3f3c00b1c0 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 10 monclient(hunting): picked mon.noname-a con 0x7f3f5c134180 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 10 monclient(hunting): picked mon.noname-e con 0x7f3f5c1d30f0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c09d420 con 0x7f3f5c134180 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c09f710 con 0x7f3f3c00b1c0 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f5a59c640 1 -- >> v1:172.21.15.7:6792/0 conn(0x7f3f3c00b1c0 legacy=0x7f3f5c133a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f5c09ee70 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f60b60640 10 monclient(hunting): authenticate will time out at 1102.388985s 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f59d9b640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f3f5c134180 0x7f3f5c1d19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41306/0 (socket says 172.21.15.7:41306) 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.274+0000 7f3f59d9b640 1 -- 172.21.15.7:0/1839141083 learned_addr learned my addr 172.21.15.7:0/1839141083 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:34.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.276+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4183644691 0 0) 0x7f3f5c09ee70 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.276+0000 7f3f4affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.276+0000 7f3f4affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.276+0000 7f3f4affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.276+0000 7f3f4affd640 10 monclient(hunting): my global_id is 94958 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f3c0053f0 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3388072722 0 0) 0x7f3f3c0053f0 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f3f5c09ee70 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3f54003cb0 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:34.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient(hunting): dump: 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:34.282 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 301850305 0 0) 0x7f3f5c09ee70 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 >> v1:172.21.15.7:6792/0 conn(0x7f3f3c00b1c0 legacy=0x7f3f5c133a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 >> v1:172.21.15.7:6789/0 conn(0x7f3f5c134180 legacy=0x7f3f5c1d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 monclient: found mon.e 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f5c09b7a0 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: _finish_auth 0 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:34.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f60b60640 5 monclient: authenticate success, global_id 94958 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f60b60640 10 monclient: _renew_subs 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f60b60640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f3f5c09fe80 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f60b60640 10 monclient: _renew_subs 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f60b60640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f3f5c1d6c00 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f3f54003f10 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f3f54004400 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f58d99640 4 set_mon_vals no callback set 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.277+0000 7f3f4affd640 10 monclient: dump: 2024-05-08T15:51:34.284 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:34.285 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.278+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f3f54004660 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.278+0000 7f3f60b60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.278+0000 7f3f60b60640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.278+0000 7f3f60b60640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.278+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(197..197 src has 1..197) ==== 3898+0+0 (unknown 2948488469 0 0) 0x7f3f54046f10 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.278+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f40005080 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.279+0000 7f3f5a59c640 10 monclient: get_auth_request con 0x7f3f3c03ae20 auth_method 0 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.284+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f3f54016040 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.284+0000 7f3f4affd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:34.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.284+0000 7f3f4affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:34.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.546+0000 7f3f60b60640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:51:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.546+0000 7f3f60b60640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.546+0000 7f3f60b60640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.546+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f3f40005370 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.547+0000 7f3f4affd640 1 -- 172.21.15.7:0/1839141083 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2684 (unknown 254746657 0 3943979597) 0x7f3f54015940 con 0x7f3f5c1d30f0 2024-05-08T15:51:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.547+0000 7f3f4affd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:34.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.547+0000 7f3f4affd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:34.543 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":34,"quorum":[1,2,3,5,7,8],"quorum_names":["f","b","g","h","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:51:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 >> v1:172.21.15.81:6812/847627118 conn(0x7f3f3c03ae20 legacy=0x7f3f3c03d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 10 monclient: shutdown 2024-05-08T15:51:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:34.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 >> v1:172.21.15.7:6793/0 conn(0x7f3f5c1d30f0 legacy=0x7f3f5c1d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 shutdown_connections 2024-05-08T15:51:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 >> 172.21.15.7:0/1839141083 conn(0x7f3f5c0a9fa0 msgr2=0x7f3f5c136950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 shutdown_connections 2024-05-08T15:51:34.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:34.549+0000 7f3f60b60640 1 -- 172.21.15.7:0/1839141083 wait complete. 2024-05-08T15:51:37.557 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.675+0000 7fd6568e0640 10 monclient: get_monmap_and_config 2024-05-08T15:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.675+0000 7fd6568e0640 10 monclient: build_initial_monmap 2024-05-08T15:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.675+0000 7fd6568e0640 10 monclient: monmap: 2024-05-08T15:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:37.676437+0000 2024-05-08T15:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:37.676437+0000 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 1 Processor -- start 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 1 -- start start 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient: init 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient: _add_conns ranks=[7,2,4,3,0,1,5,8,6] 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient(hunting): picked mon.noname-i con 0x7fd650137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient(hunting): picked mon.noname-b con 0x7fd6500a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient(hunting): picked mon.noname-c con 0x7fd6500a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6500a2c70 con 0x7fd6500a49a0 2024-05-08T15:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6500a2f50 con 0x7fd6500a41b0 2024-05-08T15:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6500a1670 con 0x7fd650137940 2024-05-08T15:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd6568e0640 10 monclient(hunting): authenticate will time out at 1105.791553s 2024-05-08T15:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.676+0000 7fd654e7d640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7fd6500a41b0 legacy=0x7fd650140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.677+0000 7fd64f7fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd6500a49a0 0x7fd65013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:60548/0 (socket says 172.21.15.7:60548) 2024-05-08T15:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.677+0000 7fd64f7fe640 1 -- 172.21.15.7:0/783053093 learned_addr learned my addr 172.21.15.7:0/783053093 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:37.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:37.877+0000 7fd654e7d640 1 -- 172.21.15.7:0/783053093 >> v1:172.21.15.7:6791/0 conn(0x7fd6500a41b0 legacy=0x7fd650140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:38.278+0000 7fd654e7d640 1 -- 172.21.15.7:0/783053093 >> v1:172.21.15.7:6791/0 conn(0x7fd6500a41b0 legacy=0x7fd650140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:39.079+0000 7fd654e7d640 1 -- 172.21.15.7:0/783053093 >> v1:172.21.15.7:6791/0 conn(0x7fd6500a41b0 legacy=0x7fd650140610 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 440394433 0 0) 0x7fd6500a1670 con 0x7fd650137940 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): my global_id is 114316 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd638003520 con 0x7fd650137940 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2719577508 0 0) 0x7fd6500a2c70 con 0x7fd6500a49a0 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 10 monclient(hunting): my global_id is 95087 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.316+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6500a1670 con 0x7fd6500a49a0 2024-05-08T15:51:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4100399709 0 0) 0x7fd638003520 con 0x7fd650137940 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 >> v1:172.21.15.7:6791/0 conn(0x7fd6500a41b0 legacy=0x7fd650140610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 >> v1:172.21.15.7:6790/0 conn(0x7fd6500a49a0 legacy=0x7fd65013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 monclient: found mon.noname-i 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.81:6792/0 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6500a39f0 con 0x7fd650137940 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: _finish_auth 0 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd640001790 con 0x7fd650137940 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd6568e0640 5 monclient: authenticate success, global_id 114316 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd6568e0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: dump: 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd6568e0640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 <== mon.7 v1:172.21.15.81:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd6400019f0 con 0x7fd650137940 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 1 -- 172.21.15.7:0/783053093 <== mon.7 v1:172.21.15.81:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd640003950 con 0x7fd650137940 2024-05-08T15:51:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd6568e0640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd6568e0640 4 set_mon_vals no callback set 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.317+0000 7fd64effd640 10 monclient: dump: 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.319 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.318+0000 7fd6568e0640 10 monclient: shutdown 2024-05-08T15:51:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.318+0000 7fd6568e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.318+0000 7fd6568e0640 1 -- 172.21.15.7:0/783053093 >> v1:172.21.15.81:6792/0 conn(0x7fd650137940 legacy=0x7fd650137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.318+0000 7fd6568e0640 1 -- 172.21.15.7:0/783053093 shutdown_connections 2024-05-08T15:51:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.318+0000 7fd6568e0640 1 -- 172.21.15.7:0/783053093 >> 172.21.15.7:0/783053093 conn(0x7fd6500a9fa0 msgr2=0x7fd6500aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.318+0000 7fd6568e0640 1 -- 172.21.15.7:0/783053093 shutdown_connections 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.318+0000 7fd6568e0640 1 -- 172.21.15.7:0/783053093 wait complete. 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.319+0000 7fd6568e0640 10 monclient: build_initial_monmap 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.319+0000 7fd6568e0640 10 monclient: monmap: 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.319+0000 7fd6568e0640 1 Processor -- start 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 1 -- start start 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 10 monclient: init 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 10 monclient: _add_conns ranks=[7,5,8,0,4,1,6,3,2] 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 10 monclient(hunting): picked mon.noname-i con 0x7fd6500a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 10 monclient(hunting): picked mon.noname-h con 0x7fd6500a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 10 monclient(hunting): picked mon.noname-e con 0x7fd650137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.320+0000 7fd6568e0640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6501cd070 con 0x7fd6500a49a0 2024-05-08T15:51:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.321+0000 7fd6568e0640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd650124360 con 0x7fd6500a41b0 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.321+0000 7fd6568e0640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd650122bd0 con 0x7fd650137940 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.321+0000 7fd6568e0640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.321+0000 7fd6568e0640 10 monclient(hunting): authenticate will time out at 1108.435658s 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.321+0000 7fd64ffff640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7fd6500a41b0 0x7fd650136bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:60810/0 (socket says 172.21.15.7:60810) 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.321+0000 7fd654e7d640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fd650137940 0x7fd6501d4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:37694/0 (socket says 172.21.15.7:37694) 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.321+0000 7fd64ffff640 1 -- 172.21.15.7:0/2310754890 learned_addr learned my addr 172.21.15.7:0/2310754890 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.324+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2001272736 0 0) 0x7fd650122bd0 con 0x7fd650137940 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.324+0000 7fd64cff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.324+0000 7fd64cff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.324+0000 7fd64cff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.324+0000 7fd64cff9640 10 monclient(hunting): my global_id is 94967 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.324+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd63c003520 con 0x7fd650137940 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4248991341 0 0) 0x7fd63c003520 con 0x7fd650137940 2024-05-08T15:51:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd650122bd0 con 0x7fd650137940 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6480046f0 con 0x7fd650137940 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient(hunting): dump: 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.327 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4036312960 0 0) 0x7fd650122bd0 con 0x7fd650137940 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 >> v1:172.21.15.81:6792/0 conn(0x7fd6500a41b0 legacy=0x7fd650136bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 >> v1:172.21.15.81:6791/0 conn(0x7fd6500a49a0 legacy=0x7fd6501d1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 monclient: found mon.e 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6500a0f60 con 0x7fd650137940 2024-05-08T15:51:40.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: _finish_auth 0 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd6568e0640 5 monclient: authenticate success, global_id 94967 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd6568e0640 10 monclient: _renew_subs 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd6568e0640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd6500a1f30 con 0x7fd650137940 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd6568e0640 10 monclient: _renew_subs 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd6568e0640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fd6501372e0 con 0x7fd650137940 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd648004cb0 con 0x7fd650137940 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd648005220 con 0x7fd650137940 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:40.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64cff9640 10 monclient: dump: 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.330 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.325+0000 7fd64e7fc640 4 set_mon_vals no callback set 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.326+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd6480053c0 con 0x7fd650137940 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.326+0000 7fd6568e0640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.326+0000 7fd6568e0640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.326+0000 7fd6568e0640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.326+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd630005080 con 0x7fd650137940 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.326+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(198..198 src has 1..198) ==== 3898+0+0 (unknown 2438200742 0 0) 0x7fd648047220 con 0x7fd650137940 2024-05-08T15:51:40.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.327+0000 7fd64ffff640 10 monclient: get_auth_request con 0x7fd63c039ad0 auth_method 0 2024-05-08T15:51:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.332+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fd648016290 con 0x7fd650137940 2024-05-08T15:51:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.332+0000 7fd64cff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:40.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.332+0000 7fd64cff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:40.449 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:40.449 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37184]: starting. 2024-05-08T15:51:40.449 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37184]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37083 2024-05-08T15:51:40.450 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37184]: created object 0... 2024-05-08T15:51:40.450 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37184]: created object 25... 2024-05-08T15:51:40.450 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37184]: created object 49... 2024-05-08T15:51:40.450 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37184]: finishing. 2024-05-08T15:51:40.450 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37184]: shutting down. 2024-05-08T15:51:40.460 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:40.460 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37186]: starting. 2024-05-08T15:51:40.460 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37186]: listing objects. 2024-05-08T15:51:40.460 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37186]: listed object 0... 2024-05-08T15:51:40.460 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37186]: listed object 25... 2024-05-08T15:51:40.460 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37186]: saw 50 objects 2024-05-08T15:51:40.460 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37186]: shutting down. 2024-05-08T15:51:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.603+0000 7fd6568e0640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:51:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.603+0000 7fd6568e0640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.603+0000 7fd6568e0640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.603+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd630005370 con 0x7fd650137940 2024-05-08T15:51:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.604+0000 7fd64cff9640 1 -- 172.21.15.7:0/2310754890 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2690 (unknown 254746657 0 1441891363) 0x7fd648015b90 con 0x7fd650137940 2024-05-08T15:51:40.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.604+0000 7fd64cff9640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:51:40.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.604+0000 7fd64cff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:40.602 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":36,"quorum":[0,1,2,3,5,7,8],"quorum_names":["a","f","b","g","h","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:51:40.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.605+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 >> v1:172.21.15.81:6812/847627118 conn(0x7fd63c039ad0 legacy=0x7fd63c03bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.605+0000 7fd6568e0640 10 monclient: shutdown 2024-05-08T15:51:40.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.605+0000 7fd6568e0640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:40.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.605+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 >> v1:172.21.15.7:6793/0 conn(0x7fd650137940 legacy=0x7fd6501d4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.606+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 shutdown_connections 2024-05-08T15:51:40.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.606+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 >> 172.21.15.7:0/2310754890 conn(0x7fd6500a9fa0 msgr2=0x7fd6500a89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:40.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.606+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 shutdown_connections 2024-05-08T15:51:40.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.606+0000 7fd6568e0640 1 -- 172.21.15.7:0/2310754890 wait complete. 2024-05-08T15:51:40.613 INFO:tasks.mon_thrash.ceph_manager:quorum is size 7 2024-05-08T15:51:40.613 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon a 2024-05-08T15:51:40.613 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:51:40.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.721+0000 7efd010f8640 10 monclient: get_monmap_and_config 2024-05-08T15:51:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.721+0000 7efd010f8640 10 monclient: build_initial_monmap 2024-05-08T15:51:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.721+0000 7efd010f8640 10 monclient: monmap: 2024-05-08T15:51:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:40.722273+0000 2024-05-08T15:51:40.718 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:40.722273+0000 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:40.719 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:40.720 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.721+0000 7efd010f8640 1 Processor -- start 2024-05-08T15:51:40.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.721+0000 7efd010f8640 1 -- start start 2024-05-08T15:51:40.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.721+0000 7efd010f8640 10 monclient: init 2024-05-08T15:51:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 10 monclient: _add_conns ranks=[1,7,2,5,8,4,0,6,3] 2024-05-08T15:51:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 10 monclient(hunting): picked mon.noname-f con 0x7efcfc0a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 10 monclient(hunting): picked mon.noname-i con 0x7efcfc0a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 10 monclient(hunting): picked mon.noname-b con 0x7efcfc13e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc0a1660 con 0x7efcfc0a53a0 2024-05-08T15:51:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc0a28f0 con 0x7efcfc13e310 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc0a2bd0 con 0x7efcfc0a5780 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efd010f8640 10 monclient(hunting): authenticate will time out at 1108.837137s 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcfb59e640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7efcfc13e310 0x7efcfc140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:60554/0 (socket says 172.21.15.7:60554) 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcfb59e640 1 -- 172.21.15.7:0/1548701874 learned_addr learned my addr 172.21.15.7:0/1548701874 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2549706487 0 0) 0x7efcfc0a28f0 con 0x7efcfc13e310 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcf9d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcf9d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcf9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcf9d9b640 10 monclient(hunting): my global_id is 95096 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.722+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efce8003520 con 0x7efcfc13e310 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4279482757 0 0) 0x7efcfc0a1660 con 0x7efcfc0a53a0 2024-05-08T15:51:40.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): my global_id is 5671 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcfc0a28f0 con 0x7efcfc0a53a0 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913807012 0 0) 0x7efcfc0a2bd0 con 0x7efcfc0a5780 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient(hunting): my global_id is 114325 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcfc0a1660 con 0x7efcfc0a5780 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1950424675 0 0) 0x7efce8003520 con 0x7efcfc13e310 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 >> v1:172.21.15.81:6792/0 conn(0x7efcfc0a5780 legacy=0x7efcfc13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 >> v1:172.21.15.81:6789/0 conn(0x7efcfc0a53a0 legacy=0x7efcfc0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: _finish_hunting 0 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 monclient: found mon.noname-b 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc0a6730 con 0x7efcfc13e310 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: _finish_auth 0 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: _check_auth_tickets 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcf4001b30 con 0x7efcfc13e310 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 5 monclient: authenticate success, global_id 95096 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: dump: 2024-05-08T15:51:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efcf4001d90 con 0x7efcfc13e310 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 1 -- 172.21.15.7:0/1548701874 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcf4004f60 con 0x7efcfc13e310 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 4 set_mon_vals no callback set 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efcf9d9b640 10 monclient: dump: 2024-05-08T15:51:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.739 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 10 monclient: shutdown 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 1 -- 172.21.15.7:0/1548701874 >> v1:172.21.15.7:6790/0 conn(0x7efcfc13e310 legacy=0x7efcfc140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 1 -- 172.21.15.7:0/1548701874 shutdown_connections 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 1 -- 172.21.15.7:0/1548701874 >> 172.21.15.7:0/1548701874 conn(0x7efcfc0a9fa0 msgr2=0x7efcfc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 1 -- 172.21.15.7:0/1548701874 shutdown_connections 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.723+0000 7efd010f8640 1 -- 172.21.15.7:0/1548701874 wait complete. 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.724+0000 7efd010f8640 10 monclient: build_initial_monmap 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.724+0000 7efd010f8640 10 monclient: monmap: 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:40.740 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.724+0000 7efd010f8640 1 Processor -- start 2024-05-08T15:51:40.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 1 -- start start 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient: init 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient: _add_conns ranks=[0,6,4,7,8,1,3,5,2] 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient(hunting): picked mon.noname-a con 0x7efcfc0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient(hunting): picked mon.noname-d con 0x7efcfc0a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient(hunting): picked mon.noname-c con 0x7efcfc13e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc135170 con 0x7efcfc0a53a0 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc132f20 con 0x7efcfc13e310 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcfc0a1e00 con 0x7efcfc0a5780 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efd010f8640 10 monclient(hunting): authenticate will time out at 1108.840100s 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcfa59c640 1 -- >> v1:172.21.15.7:6792/0 conn(0x7efcfc0a5780 legacy=0x7efcfc13c220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcfb59e640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7efcfc13e310 legacy=0x7efcfc1d4d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcfad9d640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7efcfc0a53a0 0x7efcfc13bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50716/0 (socket says 172.21.15.7:50716) 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcfad9d640 1 -- 172.21.15.7:0/4148441341 learned_addr learned my addr 172.21.15.7:0/4148441341 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 603508480 0 0) 0x7efcfc0a28f0 con 0x7efcfc0a53a0 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcf37fe640 10 monclient(hunting): my global_id is 94959 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcdc003520 con 0x7efcfc0a53a0 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.725+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1433743029 0 0) 0x7efcdc003520 con 0x7efcfc0a53a0 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efcfc0a28f0 con 0x7efcfc0a53a0 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efce4001b10 con 0x7efcfc0a53a0 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient(hunting): dump: 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.743 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1751437331 0 0) 0x7efcfc0a28f0 con 0x7efcfc0a53a0 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.7:6792/0 conn(0x7efcfc0a5780 legacy=0x7efcfc13c220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.7:6791/0 conn(0x7efcfc13e310 legacy=0x7efcfc1d4d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 monclient: found mon.a 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcfc0a43c0 con 0x7efcfc0a53a0 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 5 monclient: authenticate success, global_id 94959 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 10 monclient: _renew_subs 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:40.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7efcfc13c930 con 0x7efcfc0a53a0 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 10 monclient: _renew_subs 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7efcfc134a70 con 0x7efcfc0a53a0 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efce40038c0 con 0x7efcfc0a53a0 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efce4003e30 con 0x7efcfc0a53a0 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf959a640 4 set_mon_vals no callback set 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efcf37fe640 10 monclient: dump: 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.746 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.726+0000 7efd010f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:40.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efce00065c0 con 0x7efce0002860 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7efce4003fd0 con 0x7efcfc0a53a0 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(199..199 src has 1..199) ==== 3898+0+0 (unknown 2018725784 0 0) 0x7efce4046f40 con 0x7efcfc0a53a0 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1829001825 0 0) 0x7efce00065c0 con 0x7efce0002860 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 10 monclient(hunting): my global_id is 94968 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcfc0a28f0 con 0x7efce0002860 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcfb59e640 10 monclient: get_auth_request con 0x7efcdc039a90 auth_method 0 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1224732373 0 0) 0x7efcfc0a28f0 con 0x7efce0002860 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efce00065c0 con 0x7efce0002860 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcec001860 con 0x7efce0002860 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:40.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.727+0000 7efcf37fe640 10 monclient: dump: 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.749 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.728+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 287269977 0 0) 0x7efce00065c0 con 0x7efce0002860 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.728+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efce000a150 con 0x7efce0002860 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.728+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7efcec0057c0 con 0x7efce0002860 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.728+0000 7efcf37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.728+0000 7efcf37fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.728+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.7:6789/0 conn(0x7efce0002860 legacy=0x7efce0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.728+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.7:6789/0 conn(0x7efce0002860 legacy=0x7efce0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.734+0000 7efd010f8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.734+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efce0009e20 con 0x7efce00090e0 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3568143897 0 0) 0x7efce0009e20 con 0x7efce00090e0 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient(hunting): my global_id is 94977 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efce00065c0 con 0x7efce00090e0 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2896449754 0 0) 0x7efce00065c0 con 0x7efce00090e0 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efce0009e20 con 0x7efce00090e0 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efcec001ac0 con 0x7efce00090e0 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient: dump: 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.751 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1686817019 0 0) 0x7efce0009e20 con 0x7efce00090e0 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efce000c390 con 0x7efce00090e0 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7501 (unknown 0 0 3749872579) 0x7efcec005970 con 0x7efce00090e0 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.7:6789/0 conn(0x7efce00090e0 legacy=0x7efce0009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.735+0000 7efcf37fe640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.7:6789/0 conn(0x7efce00090e0 legacy=0x7efce0009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 36, 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 151286, 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:40.753 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:40.754 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:40.755 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:40.756 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:40.757 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.758 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:40.759 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.760 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.761 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.762 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.763 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.764 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.765 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.767 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.768 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:40.769 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.770 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:40.771 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.81:6812/847627118 conn(0x7efcdc039a90 legacy=0x7efcdc03bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 10 monclient: shutdown 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 >> v1:172.21.15.7:6789/0 conn(0x7efcfc0a53a0 legacy=0x7efcfc13bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efcfb59e640 1 -- 172.21.15.7:0/4148441341 reap_dead start 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efcfb59e640 1 -- 172.21.15.7:0/4148441341 reap_dead start 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 shutdown_connections 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 >> 172.21.15.7:0/4148441341 conn(0x7efcfc0a9fa0 msgr2=0x7efcfc0a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 shutdown_connections 2024-05-08T15:51:40.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.736+0000 7efd010f8640 1 -- 172.21.15.7:0/4148441341 wait complete. 2024-05-08T15:51:40.773 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:51:40.774 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon b 2024-05-08T15:51:40.774 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:51:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.881+0000 7f2651f23640 10 monclient: get_monmap_and_config 2024-05-08T15:51:40.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.881+0000 7f2651f23640 10 monclient: build_initial_monmap 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.881+0000 7f2651f23640 10 monclient: monmap: 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:40.882545+0000 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:40.882545+0000 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:40.878 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:40.879 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:40.879 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:40.879 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:40.879 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.882+0000 7f2651f23640 1 Processor -- start 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.882+0000 7f2651f23640 1 -- start start 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.882+0000 7f2651f23640 10 monclient: init 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.882+0000 7f2651f23640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.882+0000 7f2651f23640 10 monclient: _add_conns ranks=[4,8,5,2,1,0,3,7,6] 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.882+0000 7f2651f23640 10 monclient(hunting): picked mon.noname-c con 0x7f264c137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.882+0000 7f2651f23640 10 monclient(hunting): picked mon.noname-e con 0x7f264c0a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 10 monclient(hunting): picked mon.noname-h con 0x7f264c0a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0a2c70 con 0x7f264c137940 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0a2f50 con 0x7f264c0a41b0 2024-05-08T15:51:40.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0a1670 con 0x7f264c0a49a0 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 10 monclient(hunting): authenticate will time out at 1108.997645s 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264b7fe640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7f264c137940 legacy=0x7f264c137d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2643fff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f264c0a49a0 0x7f264c13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:37710/0 (socket says 172.21.15.7:37710) 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2643fff640 1 -- 172.21.15.7:0/3383297865 learned_addr learned my addr 172.21.15.7:0/3383297865 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1493303617 0 0) 0x7f264c0a1670 con 0x7f264c0a49a0 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): my global_id is 94976 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2638003520 con 0x7f264c0a49a0 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573026595 0 0) 0x7f264c0a2f50 con 0x7f264c0a41b0 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient(hunting): my global_id is 5738 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f264c0a1670 con 0x7f264c0a41b0 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4095895552 0 0) 0x7f264c0a1670 con 0x7f264c0a41b0 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 >> v1:172.21.15.7:6793/0 conn(0x7f264c0a49a0 legacy=0x7f264c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 >> v1:172.21.15.7:6791/0 conn(0x7f264c137940 legacy=0x7f264c137d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: _finish_hunting 0 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 monclient: found mon.noname-h 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c0a39f0 con 0x7f264c0a41b0 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: _finish_auth 0 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: _check_auth_tickets 2024-05-08T15:51:40.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2644001b10 con 0x7f264c0a41b0 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: dump: 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.891 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 5 monclient: authenticate success, global_id 5738 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2644001df0 con 0x7f264c0a41b0 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 1 -- 172.21.15.7:0/3383297865 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2644003cd0 con 0x7f264c0a41b0 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f2651f23640 4 set_mon_vals no callback set 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.883+0000 7f264affd640 10 monclient: dump: 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.893 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 10 monclient: shutdown 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 1 -- 172.21.15.7:0/3383297865 >> v1:172.21.15.81:6791/0 conn(0x7f264c0a41b0 legacy=0x7f264c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 1 -- 172.21.15.7:0/3383297865 shutdown_connections 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 1 -- 172.21.15.7:0/3383297865 >> 172.21.15.7:0/3383297865 conn(0x7f264c0a9fa0 msgr2=0x7f264c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 1 -- 172.21.15.7:0/3383297865 shutdown_connections 2024-05-08T15:51:40.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 1 -- 172.21.15.7:0/3383297865 wait complete. 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 10 monclient: build_initial_monmap 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.884+0000 7f2651f23640 10 monclient: monmap: 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:40.895 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 1 Processor -- start 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 1 -- start start 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient: init 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:40.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient: _add_conns ranks=[7,5,3,1,4,2,8,0,6] 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient(hunting): picked mon.noname-i con 0x7f264c0a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient(hunting): picked mon.noname-h con 0x7f264c0a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient(hunting): picked mon.noname-g con 0x7f264c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0a1f30 con 0x7f264c137940 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c0a0f60 con 0x7f264c0a49a0 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264c136470 con 0x7f264c0a41b0 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.885+0000 7f2651f23640 10 monclient(hunting): authenticate will time out at 1109.000396s 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2643fff640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7f264c0a49a0 0x7f264c134f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:53434/0 (socket says 172.21.15.7:53434) 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2643fff640 1 -- 172.21.15.7:0/3933032773 learned_addr learned my addr 172.21.15.7:0/3933032773 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f264bfff640 1 --1- 172.21.15.7:0/3933032773 >> v1:172.21.15.81:6790/0 conn(0x7f264c137940 0x7f264c1d4340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:53150/0 (socket says 172.21.15.7:53150) 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465456468 0 0) 0x7f264c136470 con 0x7f264c0a41b0 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): my global_id is 114334 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f262c003520 con 0x7f264c0a41b0 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2702755705 0 0) 0x7f264c0a0f60 con 0x7f264c0a49a0 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): my global_id is 5747 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f264c136470 con 0x7f264c0a49a0 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 236127019 0 0) 0x7f264c0a1f30 con 0x7f264c137940 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): my global_id is 75990 2024-05-08T15:51:40.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f264c0a0f60 con 0x7f264c137940 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2900229505 0 0) 0x7f262c003520 con 0x7f264c0a41b0 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f264c0a1f30 con 0x7f264c0a41b0 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f263c004190 con 0x7f264c0a41b0 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.900 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1689245904 0 0) 0x7f264c136470 con 0x7f264c0a49a0 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f262c003520 con 0x7f264c0a49a0 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1873359064 0 0) 0x7f264c0a0f60 con 0x7f264c137940 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f264c136470 con 0x7f264c137940 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2634001550 con 0x7f264c0a49a0 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:40.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.902 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2644003d50 con 0x7f264c137940 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.903 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1228324387 0 0) 0x7f264c0a1f30 con 0x7f264c0a41b0 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.81:6791/0 conn(0x7f264c0a49a0 legacy=0x7f264c134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.81:6790/0 conn(0x7f264c137940 legacy=0x7f264c1d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 monclient: found mon.i 2024-05-08T15:51:40.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264c135690 con 0x7f264c0a41b0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: _finish_auth 0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2651f23640 5 monclient: authenticate success, global_id 114334 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2651f23640 10 monclient: _renew_subs 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2651f23640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f264c139570 con 0x7f264c0a41b0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2651f23640 10 monclient: _renew_subs 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2651f23640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f264c136d10 con 0x7f264c0a41b0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f263c0044b0 con 0x7f264c0a41b0 2024-05-08T15:51:40.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f263c004a20 con 0x7f264c0a41b0 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f264a7fc640 4 set_mon_vals no callback set 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.886+0000 7f2648ff9640 10 monclient: dump: 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.906 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.887+0000 7f2651f23640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.887+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f26300065c0 con 0x7f2630002860 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.887+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f263c004c40 con 0x7f264c0a41b0 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.887+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(199..199 src has 1..199) ==== 3898+0+0 (unknown 2018725784 0 0) 0x7f263c046e10 con 0x7f264c0a41b0 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f264bfff640 10 monclient: get_auth_request con 0x7f262c03a550 auth_method 0 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650379268 0 0) 0x7f26300065c0 con 0x7f2630002860 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient(hunting): my global_id is 95105 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f264c0a1f30 con 0x7f2630002860 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1682255621 0 0) 0x7f264c0a1f30 con 0x7f2630002860 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26300065c0 con 0x7f2630002860 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f26340017f0 con 0x7f2630002860 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient: dump: 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.908 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2872985986 0 0) 0x7f26300065c0 con 0x7f2630002860 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f263000a150 con 0x7f2630002860 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2634004f60 con 0x7f2630002860 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:40.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.7:6790/0 conn(0x7f2630002860 legacy=0x7f2630004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.888+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.7:6790/0 conn(0x7f2630002860 legacy=0x7f2630004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:40.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2651f23640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:40.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2630009e00 con 0x7f26300090e0 2024-05-08T15:51:40.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1592682824 0 0) 0x7f2630009e00 con 0x7f26300090e0 2024-05-08T15:51:40.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:40.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 10 monclient(hunting): my global_id is 95114 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26300065c0 con 0x7f26300090e0 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3468269285 0 0) 0x7f26300065c0 con 0x7f26300090e0 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2630009e00 con 0x7f26300090e0 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f26340058f0 con 0x7f26300090e0 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 10 monclient: dump: 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:40.912 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4175063145 0 0) 0x7f2630009e00 con 0x7f26300090e0 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.895+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f263000c380 con 0x7f26300090e0 2024-05-08T15:51:40.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7501 (unknown 0 0 1382350662) 0x7f2634006cb0 con 0x7f26300090e0 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2648ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2648ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.7:6790/0 conn(0x7f26300090e0 legacy=0x7f2630009550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2648ff9640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.7:6790/0 conn(0x7f26300090e0 legacy=0x7f2630009550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 36, 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 151446, 2024-05-08T15:51:40.914 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:40.915 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:40.916 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:40.917 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:40.918 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.919 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.920 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.921 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:40.922 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.923 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:40.924 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.925 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:40.926 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.927 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.928 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.929 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.930 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:40.931 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:40.932 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.81:6812/847627118 conn(0x7f262c03a550 legacy=0x7f262c03ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 10 monclient: shutdown 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 >> v1:172.21.15.81:6792/0 conn(0x7f264c0a41b0 legacy=0x7f264c134870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f264bfff640 1 -- 172.21.15.7:0/3933032773 reap_dead start 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f264bfff640 1 -- 172.21.15.7:0/3933032773 reap_dead start 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 shutdown_connections 2024-05-08T15:51:40.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 >> 172.21.15.7:0/3933032773 conn(0x7f264c0a9fa0 msgr2=0x7f264c0a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 shutdown_connections 2024-05-08T15:51:40.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:40.896+0000 7f2651f23640 1 -- 172.21.15.7:0/3933032773 wait complete. 2024-05-08T15:51:40.934 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:51:40.934 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon e 2024-05-08T15:51:40.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:51:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.041+0000 7faca6674640 10 monclient: get_monmap_and_config 2024-05-08T15:51:41.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.041+0000 7faca6674640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.041+0000 7faca6674640 10 monclient: monmap: 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:41.042723+0000 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:41.042723+0000 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.038 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.039 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.039 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.039 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.042+0000 7faca6674640 1 Processor -- start 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.042+0000 7faca6674640 1 -- start start 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.042+0000 7faca6674640 10 monclient: init 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient: _add_conns ranks=[4,2,5,0,6,8,3,7,1] 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient(hunting): picked mon.noname-c con 0x7faca0137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient(hunting): picked mon.noname-b con 0x7faca00a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient(hunting): picked mon.noname-h con 0x7faca00a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:41.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca00a2c70 con 0x7faca00a49a0 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca00a2f50 con 0x7faca0137940 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca00a1670 con 0x7faca00a41b0 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient(hunting): authenticate will time out at 1109.157830s 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9ffff640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7faca0137940 legacy=0x7faca0137d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac97fff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7faca00a49a0 0x7faca013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:60584/0 (socket says 172.21.15.7:60584) 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac97fff640 1 -- 172.21.15.7:0/2293064087 learned_addr learned my addr 172.21.15.7:0/2293064087 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 265032601 0 0) 0x7faca00a2c70 con 0x7faca00a49a0 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): my global_id is 95123 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac8c003520 con 0x7faca00a49a0 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577209810 0 0) 0x7faca00a1670 con 0x7faca00a41b0 2024-05-08T15:51:41.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient(hunting): my global_id is 5756 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faca00a2c70 con 0x7faca00a41b0 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 666082798 0 0) 0x7fac8c003520 con 0x7faca00a49a0 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 >> v1:172.21.15.81:6791/0 conn(0x7faca00a41b0 legacy=0x7faca0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 >> v1:172.21.15.7:6791/0 conn(0x7faca0137940 legacy=0x7faca0137d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 monclient: found mon.noname-b 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca00a39f0 con 0x7faca00a49a0 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fac880017b0 con 0x7faca00a49a0 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 5 monclient: authenticate success, global_id 95123 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: dump: 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.049 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fac88001ad0 con 0x7faca00a49a0 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 1 -- 172.21.15.7:0/2293064087 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fac88004be0 con 0x7faca00a49a0 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7faca6674640 4 set_mon_vals no callback set 2024-05-08T15:51:41.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.043+0000 7fac9f7fe640 10 monclient: dump: 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.051 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 10 monclient: shutdown 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 1 -- 172.21.15.7:0/2293064087 >> v1:172.21.15.7:6790/0 conn(0x7faca00a49a0 legacy=0x7faca013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 1 -- 172.21.15.7:0/2293064087 shutdown_connections 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 1 -- 172.21.15.7:0/2293064087 >> 172.21.15.7:0/2293064087 conn(0x7faca00a9fa0 msgr2=0x7faca00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 1 -- 172.21.15.7:0/2293064087 shutdown_connections 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 1 -- 172.21.15.7:0/2293064087 wait complete. 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.044+0000 7faca6674640 10 monclient: monmap: 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:41.052 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 1 Processor -- start 2024-05-08T15:51:41.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 1 -- start start 2024-05-08T15:51:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient: init 2024-05-08T15:51:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient: _add_conns ranks=[7,2,4,0,1,5,3,6,8] 2024-05-08T15:51:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient(hunting): picked mon.noname-i con 0x7faca00a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient(hunting): picked mon.noname-b con 0x7faca00a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient(hunting): picked mon.noname-c con 0x7faca0137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:41.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca01cda40 con 0x7faca00a49a0 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca00a1f30 con 0x7faca0137940 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faca00a0f60 con 0x7faca00a41b0 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca6674640 10 monclient(hunting): authenticate will time out at 1109.160429s 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7faca4c11640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7faca0137940 legacy=0x7faca01d4e80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7fac97fff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7faca00a49a0 0x7faca01cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:60592/0 (socket says 172.21.15.7:60592) 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.045+0000 7fac97fff640 1 -- 172.21.15.7:0/4073857514 learned_addr learned my addr 172.21.15.7:0/4073857514 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745882728 0 0) 0x7faca01cda40 con 0x7faca00a49a0 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): my global_id is 95132 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac80003520 con 0x7faca00a49a0 2024-05-08T15:51:41.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3879639907 0 0) 0x7fac80003520 con 0x7faca00a49a0 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7faca01cda40 con 0x7faca00a49a0 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fac88004b20 con 0x7faca00a49a0 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): dump: 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.056 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214809858 0 0) 0x7faca00a0f60 con 0x7faca00a41b0 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient(hunting): my global_id is 114343 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac80003520 con 0x7faca00a41b0 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3135074289 0 0) 0x7faca01cda40 con 0x7faca00a49a0 2024-05-08T15:51:41.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.81:6792/0 conn(0x7faca00a41b0 legacy=0x7faca01ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.7:6791/0 conn(0x7faca0137940 legacy=0x7faca01d4e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 monclient: found mon.b 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca0133c80 con 0x7faca00a49a0 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7faca6674640 5 monclient: authenticate success, global_id 95132 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7faca6674640 10 monclient: _renew_subs 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7faca6674640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7faca0135700 con 0x7faca00a49a0 2024-05-08T15:51:41.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7faca6674640 10 monclient: _renew_subs 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7faca6674640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7faca0136790 con 0x7faca00a49a0 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fac88001c80 con 0x7faca00a49a0 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fac88001ad0 con 0x7faca00a49a0 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9e7fc640 4 set_mon_vals no callback set 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.046+0000 7fac9d7fa640 10 monclient: dump: 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.059 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7faca6674640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fac840065c0 con 0x7fac84002860 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fac880106b0 con 0x7faca00a49a0 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(199..199 src has 1..199) ==== 3898+0+0 (unknown 2018725784 0 0) 0x7fac88046fb0 con 0x7faca00a49a0 2024-05-08T15:51:41.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1520558086 0 0) 0x7fac840065c0 con 0x7fac84002860 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 10 monclient(hunting): my global_id is 94985 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faca01cda40 con 0x7fac84002860 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.047+0000 7faca4c11640 10 monclient: get_auth_request con 0x7fac80039c50 auth_method 0 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2264314427 0 0) 0x7faca01cda40 con 0x7fac84002860 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fac840065c0 con 0x7fac84002860 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fac900041c0 con 0x7fac84002860 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 10 monclient: dump: 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.061 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 896204024 0 0) 0x7fac840065c0 con 0x7fac84002860 2024-05-08T15:51:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac8400a150 con 0x7fac84002860 2024-05-08T15:51:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fac900065d0 con 0x7fac84002860 2024-05-08T15:51:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.7:6793/0 conn(0x7fac84002860 legacy=0x7fac84004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.048+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.7:6793/0 conn(0x7fac84002860 legacy=0x7fac84004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7faca6674640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fac84009e20 con 0x7fac840090e0 2024-05-08T15:51:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379238905 0 0) 0x7fac84009e20 con 0x7fac840090e0 2024-05-08T15:51:41.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient(hunting): my global_id is 94994 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac840065c0 con 0x7fac840090e0 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2340742574 0 0) 0x7fac840065c0 con 0x7fac840090e0 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fac84009e20 con 0x7fac840090e0 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fac90004c20 con 0x7fac840090e0 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient: dump: 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.065 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3221747230 0 0) 0x7fac84009e20 con 0x7fac840090e0 2024-05-08T15:51:41.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fac8400c390 con 0x7fac840090e0 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7671 (unknown 0 0 3500997764) 0x7fac90006070 con 0x7fac840090e0 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.7:6793/0 conn(0x7fac840090e0 legacy=0x7fac84009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.055+0000 7fac9d7fa640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.7:6793/0 conn(0x7fac840090e0 legacy=0x7fac84009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 36, 2024-05-08T15:51:41.067 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 151607, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.068 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.069 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:41.070 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.071 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:41.072 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.073 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.074 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.075 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:41.076 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.077 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.078 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.079 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.080 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.081 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.082 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.083 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.084 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.085 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.086 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.086 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:41.086 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.81:6812/847627118 conn(0x7fac80039c50 legacy=0x7fac8003c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 10 monclient: shutdown 2024-05-08T15:51:41.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 >> v1:172.21.15.7:6790/0 conn(0x7faca00a49a0 legacy=0x7faca01cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca4c11640 1 -- 172.21.15.7:0/4073857514 reap_dead start 2024-05-08T15:51:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca4c11640 1 -- 172.21.15.7:0/4073857514 reap_dead start 2024-05-08T15:51:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 shutdown_connections 2024-05-08T15:51:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 >> 172.21.15.7:0/4073857514 conn(0x7faca00a9fa0 msgr2=0x7faca0130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 shutdown_connections 2024-05-08T15:51:41.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.056+0000 7faca6674640 1 -- 172.21.15.7:0/4073857514 wait complete. 2024-05-08T15:51:41.088 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:51:41.088 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon f 2024-05-08T15:51:41.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.196+0000 7fc9c72d9640 10 monclient: get_monmap_and_config 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.196+0000 7fc9c72d9640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.196+0000 7fc9c72d9640 10 monclient: monmap: 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:41.197144+0000 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:41.197144+0000 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.192 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.193 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.196+0000 7fc9c72d9640 1 Processor -- start 2024-05-08T15:51:41.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.196+0000 7fc9c72d9640 1 -- start start 2024-05-08T15:51:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.196+0000 7fc9c72d9640 10 monclient: init 2024-05-08T15:51:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 10 monclient: _add_conns ranks=[5,4,7,1,6,2,0,3,8] 2024-05-08T15:51:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 10 monclient(hunting): picked mon.noname-h con 0x7fc9c00a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:41.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 10 monclient(hunting): picked mon.noname-c con 0x7fc9c00a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 10 monclient(hunting): picked mon.noname-i con 0x7fc9c013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c00a1660 con 0x7fc9c00a5780 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c00a28f0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c00a2bd0 con 0x7fc9c013e310 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c72d9640 10 monclient(hunting): authenticate will time out at 1109.312079s 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c4874640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7fc9c00a5780 legacy=0x7fc9c013cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c5075640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fc9c00a53a0 0x7fc9c00a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:53448/0 (socket says 172.21.15.7:53448) 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.197+0000 7fc9c5075640 1 -- 172.21.15.7:0/3959012889 learned_addr learned my addr 172.21.15.7:0/3959012889 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601995812 0 0) 0x7fc9c00a28f0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): my global_id is 5765 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9ac003520 con 0x7fc9c00a53a0 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3638340939 0 0) 0x7fc9c00a2bd0 con 0x7fc9c013e310 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient(hunting): my global_id is 114352 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c00a28f0 con 0x7fc9c013e310 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4232845285 0 0) 0x7fc9ac003520 con 0x7fc9c00a53a0 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 >> v1:172.21.15.81:6792/0 conn(0x7fc9c013e310 legacy=0x7fc9c0140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 >> v1:172.21.15.7:6791/0 conn(0x7fc9c00a5780 legacy=0x7fc9c013cbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 monclient: found mon.noname-h 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: _send_mon_message to mon.noname-h at v1:172.21.15.81:6791/0 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 --> v1:172.21.15.81:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c00a6730 con 0x7fc9c00a53a0 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 5 monclient: authenticate success, global_id 5765 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9bc0017b0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: got monmap 1 from mon.noname-h (according to old e1) 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: dump: 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.205 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 <== mon.5 v1:172.21.15.81:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc9bc001a10 con 0x7fc9c00a53a0 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 4 set_mon_vals no callback set 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 1 -- 172.21.15.7:0/3959012889 <== mon.5 v1:172.21.15.81:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9bc004be0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9b3fff640 10 monclient: dump: 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.207 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 10 monclient: shutdown 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 1 -- 172.21.15.7:0/3959012889 >> v1:172.21.15.81:6791/0 conn(0x7fc9c00a53a0 legacy=0x7fc9c00a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 1 -- 172.21.15.7:0/3959012889 shutdown_connections 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.198+0000 7fc9c72d9640 1 -- 172.21.15.7:0/3959012889 >> 172.21.15.7:0/3959012889 conn(0x7fc9c00a9fa0 msgr2=0x7fc9c00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.199+0000 7fc9c72d9640 1 -- 172.21.15.7:0/3959012889 shutdown_connections 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.199+0000 7fc9c72d9640 1 -- 172.21.15.7:0/3959012889 wait complete. 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.199+0000 7fc9c72d9640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.199+0000 7fc9c72d9640 10 monclient: monmap: 2024-05-08T15:51:41.208 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.209 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.199+0000 7fc9c72d9640 1 Processor -- start 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.199+0000 7fc9c72d9640 1 -- start start 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.199+0000 7fc9c72d9640 10 monclient: init 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 10 monclient: _add_conns ranks=[5,8,3,2,4,0,6,7,1] 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 10 monclient(hunting): picked mon.noname-h con 0x7fc9c00a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 10 monclient(hunting): picked mon.noname-e con 0x7fc9c00a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 10 monclient(hunting): picked mon.noname-g con 0x7fc9c013e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c013be30 con 0x7fc9c013e310 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c00a43c0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c013aff0 con 0x7fc9c00a5780 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c72d9640 10 monclient(hunting): authenticate will time out at 1109.315035s 2024-05-08T15:51:41.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c4874640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fc9c00a5780 0x7fc9c01d1480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:37732/0 (socket says 172.21.15.7:37732) 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9c4874640 1 -- 172.21.15.7:0/4061452803 learned_addr learned my addr 172.21.15.7:0/4061452803 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3441847021 0 0) 0x7fc9c013aff0 con 0x7fc9c00a5780 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): my global_id is 95003 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9a4003520 con 0x7fc9c00a5780 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3922904016 0 0) 0x7fc9c00a43c0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): my global_id is 5774 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c013aff0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1516454101 0 0) 0x7fc9c013be30 con 0x7fc9c013e310 2024-05-08T15:51:41.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 10 monclient(hunting): my global_id is 75999 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.200+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c00a43c0 con 0x7fc9c013e310 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 453692085 0 0) 0x7fc9a4003520 con 0x7fc9c00a5780 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc9c013be30 con 0x7fc9c00a5780 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9b4004330 con 0x7fc9c00a5780 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): dump: 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.212 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2827820519 0 0) 0x7fc9c013aff0 con 0x7fc9c00a53a0 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc9a4003520 con 0x7fc9c00a53a0 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9bc004b20 con 0x7fc9c00a53a0 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): dump: 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.214 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 961358656 0 0) 0x7fc9c00a43c0 con 0x7fc9c013e310 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc9c013aff0 con 0x7fc9c013e310 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9b8001b60 con 0x7fc9c013e310 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient(hunting): dump: 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.215 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 283657343 0 0) 0x7fc9c013be30 con 0x7fc9c00a5780 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.81:6791/0 conn(0x7fc9c00a53a0 legacy=0x7fc9c013d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.81:6790/0 conn(0x7fc9c013e310 legacy=0x7fc9c01d4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 monclient: found mon.e 2024-05-08T15:51:41.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c0139e20 con 0x7fc9c00a5780 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9c72d9640 5 monclient: authenticate success, global_id 95003 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9c72d9640 10 monclient: _renew_subs 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9c72d9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c013b870 con 0x7fc9c00a5780 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fc9b4004600 con 0x7fc9c00a5780 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9b4004af0 con 0x7fc9c00a5780 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b1ffb640 10 monclient: dump: 2024-05-08T15:51:41.217 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.218 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9c72d9640 10 monclient: _renew_subs 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9c72d9640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fc9c013dd50 con 0x7fc9c00a5780 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.201+0000 7fc9b37fe640 4 set_mon_vals no callback set 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.202+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fc9b4004d10 con 0x7fc9c00a5780 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.202+0000 7fc9c72d9640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.202+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc9c0075820 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.202+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(199..199 src has 1..199) ==== 3898+0+0 (unknown 2018725784 0 0) 0x7fc9b4046f00 con 0x7fc9c00a5780 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.202+0000 7fc9c5075640 10 monclient: get_auth_request con 0x7fc9a403a4f0 auth_method 0 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 202978869 0 0) 0x7fc9c0075820 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient(hunting): my global_id is 5680 2024-05-08T15:51:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c013be30 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3197927131 0 0) 0x7fc9c013be30 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc9c0075820 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9b8003cb0 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient: dump: 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.220 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 403843453 0 0) 0x7fc9c0075820 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9c013ba40 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fc9b8003cb0 con 0x7fc9c01cc3d0 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.81:6789/0 conn(0x7fc9c01cc3d0 legacy=0x7fc9c013f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.203+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.81:6789/0 conn(0x7fc9c01cc3d0 legacy=0x7fc9c013f830 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9c72d9640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fc9c006b380 con 0x7fc9c002d050 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78764258 0 0) 0x7fc9c006b380 con 0x7fc9c002d050 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 10 monclient(hunting): my global_id is 5689 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c0075820 con 0x7fc9c002d050 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 351175029 0 0) 0x7fc9c0075820 con 0x7fc9c002d050 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fc9c006b380 con 0x7fc9c002d050 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fc9b80045d0 con 0x7fc9c002d050 2024-05-08T15:51:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.210+0000 7fc9b1ffb640 10 monclient: dump: 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2309792386 0 0) 0x7fc9c006b380 con 0x7fc9c002d050 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc9c01cc7b0 con 0x7fc9c002d050 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7843 (unknown 0 0 3794819768) 0x7fc9b8004a20 con 0x7fc9c002d050 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9b1ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9b1ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.81:6789/0 conn(0x7fc9c002d050 legacy=0x7fc9c013ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9b1ffb640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.81:6789/0 conn(0x7fc9c002d050 legacy=0x7fc9c013ff40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:41.225 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 36, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 293527, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:51:41.226 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.227 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.228 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:41.229 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:41.230 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.231 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.232 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:41.233 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.234 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:41.235 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.236 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.237 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.238 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.239 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.240 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.241 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.242 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:41.243 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:41.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.81:6812/847627118 conn(0x7fc9a403a4f0 legacy=0x7fc9a403c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9c72d9640 10 monclient: shutdown 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9c72d9640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 >> v1:172.21.15.7:6793/0 conn(0x7fc9c00a5780 legacy=0x7fc9c01d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9c5876640 1 -- 172.21.15.7:0/4061452803 reap_dead start 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 shutdown_connections 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.211+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 >> 172.21.15.7:0/4061452803 conn(0x7fc9c00a9fa0 msgr2=0x7fc9c00a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.212+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 shutdown_connections 2024-05-08T15:51:41.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.212+0000 7fc9c72d9640 1 -- 172.21.15.7:0/4061452803 wait complete. 2024-05-08T15:51:41.245 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:51:41.245 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon g 2024-05-08T15:51:41.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.354+0000 7f73bfb4f640 10 monclient: get_monmap_and_config 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.354+0000 7f73bfb4f640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.354+0000 7f73bfb4f640 10 monclient: monmap: 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:41.355489+0000 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:41.355489+0000 2024-05-08T15:51:41.358 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 1 Processor -- start 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 1 -- start start 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient: init 2024-05-08T15:51:41.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient: _add_conns ranks=[0,3,8,7,1,4,6,2,5] 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient(hunting): picked mon.noname-a con 0x7f73b8137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient(hunting): picked mon.noname-g con 0x7f73b8138430 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient(hunting): picked mon.noname-e con 0x7f73b80a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80a2990 con 0x7f73b8137940 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80a2c70 con 0x7f73b8138430 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80a39f0 con 0x7f73b80a49a0 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bfb4f640 10 monclient(hunting): authenticate will time out at 1109.470443s 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bd8eb640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f73b8137940 0x7f73b8137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50742/0 (socket says 172.21.15.7:50742) 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73bd8eb640 1 -- 172.21.15.7:0/2255396041 learned_addr learned my addr 172.21.15.7:0/2255396041 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.355+0000 7f73be0ec640 1 --1- 172.21.15.7:0/2255396041 >> v1:172.21.15.7:6793/0 conn(0x7f73b80a49a0 0x7f73b8140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:37746/0 (socket says 172.21.15.7:37746) 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 280900478 0 0) 0x7f73b4001110 con 0x7f73b8137940 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): my global_id is 94986 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a4003520 con 0x7f73b8137940 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1050404423 0 0) 0x7f73b80a39f0 con 0x7f73b80a49a0 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): my global_id is 95012 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b4001110 con 0x7f73b80a49a0 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3759431420 0 0) 0x7f73b80a2c70 con 0x7f73b8138430 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient(hunting): my global_id is 76008 2024-05-08T15:51:41.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b80a39f0 con 0x7f73b8138430 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3226527685 0 0) 0x7f73a4003520 con 0x7f73b8137940 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 >> v1:172.21.15.7:6793/0 conn(0x7f73b80a49a0 legacy=0x7f73b8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 >> v1:172.21.15.81:6790/0 conn(0x7f73b8138430 legacy=0x7f73b813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 monclient: found mon.noname-a 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b80a4290 con 0x7f73b8137940 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f73b4001980 con 0x7f73b8137940 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bfb4f640 5 monclient: authenticate success, global_id 94986 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bfb4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: dump: 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.363 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bfb4f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73b4001c20 con 0x7f73b8137940 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 1 -- 172.21.15.7:0/2255396041 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f73b4004be0 con 0x7f73b8137940 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bfb4f640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bfb4f640 4 set_mon_vals no callback set 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.356+0000 7f73bc8e9640 10 monclient: dump: 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.364 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 10 monclient: shutdown 2024-05-08T15:51:41.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 1 -- 172.21.15.7:0/2255396041 >> v1:172.21.15.7:6789/0 conn(0x7f73b8137940 legacy=0x7f73b8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 1 -- 172.21.15.7:0/2255396041 shutdown_connections 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 1 -- 172.21.15.7:0/2255396041 >> 172.21.15.7:0/2255396041 conn(0x7f73b80a9fa0 msgr2=0x7f73b80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 1 -- 172.21.15.7:0/2255396041 shutdown_connections 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 1 -- 172.21.15.7:0/2255396041 wait complete. 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.357+0000 7f73bfb4f640 10 monclient: monmap: 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.366 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 1 Processor -- start 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 1 -- start start 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient: init 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient: _add_conns ranks=[6,8,7,2,5,3,1,4,0] 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient(hunting): picked mon.noname-d con 0x7f73b80a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:41.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient(hunting): picked mon.noname-e con 0x7f73b8137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient(hunting): picked mon.noname-i con 0x7f73b8138430 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b80a1f30 con 0x7f73b80a49a0 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b81399c0 con 0x7f73b8138430 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bd8eb640 1 -- >> v1:172.21.15.7:6792/0 conn(0x7f73b80a49a0 legacy=0x7f73b81cd130 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73b81cdf50 con 0x7f73b8137940 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bfb4f640 10 monclient(hunting): authenticate will time out at 1109.473191s 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bd0ea640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f73b8137940 0x7f73b81cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:37760/0 (socket says 172.21.15.7:37760) 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73bd0ea640 1 -- 172.21.15.7:0/4086935272 learned_addr learned my addr 172.21.15.7:0/4086935272 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3067740773 0 0) 0x7f73b81cdf50 con 0x7f73b8137940 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73ae7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73ae7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73ae7fc640 10 monclient(hunting): my global_id is 95021 2024-05-08T15:51:41.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.358+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f739c003520 con 0x7f73b8137940 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3736069511 0 0) 0x7f739c003520 con 0x7f73b8137940 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73b81cdf50 con 0x7f73b8137940 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f73a8001a30 con 0x7f73b8137940 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): dump: 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.369 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3062739408 0 0) 0x7f73b81399c0 con 0x7f73b8138430 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient(hunting): my global_id is 114361 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f739c003520 con 0x7f73b8138430 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3981321326 0 0) 0x7f73b81cdf50 con 0x7f73b8137940 2024-05-08T15:51:41.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.81:6792/0 conn(0x7f73b8138430 legacy=0x7f73b81d5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.7:6792/0 conn(0x7f73b80a49a0 legacy=0x7f73b81cd130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 monclient: found mon.e 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73b8136ab0 con 0x7f73b8137940 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73bfb4f640 5 monclient: authenticate success, global_id 95021 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73bfb4f640 10 monclient: _renew_subs 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73bfb4f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f73b8139400 con 0x7f73b8137940 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73bfb4f640 10 monclient: _renew_subs 2024-05-08T15:51:41.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73bfb4f640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f73b8135ef0 con 0x7f73b8137940 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f73a8003940 con 0x7f73b8137940 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f73a8003eb0 con 0x7f73b8137940 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73affff640 4 set_mon_vals no callback set 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.359+0000 7f73ae7fc640 10 monclient: dump: 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.372 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73bfb4f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f73a00065c0 con 0x7f73a0002860 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f73a80040d0 con 0x7f73b8137940 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(199..199 src has 1..199) ==== 3898+0+0 (unknown 2018725784 0 0) 0x7f73a8046f20 con 0x7f73b8137940 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1839125886 0 0) 0x7f73a00065c0 con 0x7f73a0002860 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 10 monclient(hunting): my global_id is 76017 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73b81cdf50 con 0x7f73a0002860 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73be0ec640 10 monclient: get_auth_request con 0x7f739c039a90 auth_method 0 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1320112075 0 0) 0x7f73b81cdf50 con 0x7f73a0002860 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73a00065c0 con 0x7f73a0002860 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f73b4001940 con 0x7f73a0002860 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.360+0000 7f73ae7fc640 10 monclient: dump: 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.374 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.361+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2417667352 0 0) 0x7f73a00065c0 con 0x7f73a0002860 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.361+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f73a000a150 con 0x7f73a0002860 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.361+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f73b4001940 con 0x7f73a0002860 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.361+0000 7f73ae7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.361+0000 7f73ae7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:41.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.361+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.81:6790/0 conn(0x7f73a0002860 legacy=0x7f73a0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.361+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.81:6790/0 conn(0x7f73a0002860 legacy=0x7f73a0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.367+0000 7f73bfb4f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.367+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f73a0009e20 con 0x7f73a00090e0 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669775055 0 0) 0x7f73a0009e20 con 0x7f73a00090e0 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 10 monclient(hunting): my global_id is 76026 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73a00065c0 con 0x7f73a00090e0 2024-05-08T15:51:41.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3925891483 0 0) 0x7f73a00065c0 con 0x7f73a00090e0 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f73a0009e20 con 0x7f73a00090e0 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f73b4004c20 con 0x7f73a00090e0 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 10 monclient: dump: 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.378 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1409952972 0 0) 0x7f73a0009e20 con 0x7f73a00090e0 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.368+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f73a000c390 con 0x7f73a00090e0 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7671 (unknown 0 0 4181217730) 0x7f73b40073e0 con 0x7f73a00090e0 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73ae7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73ae7fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.81:6790/0 conn(0x7f73a00090e0 legacy=0x7f73a0009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73ae7fc640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.81:6790/0 conn(0x7f73a00090e0 legacy=0x7f73a0009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 36, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 293658, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:41.380 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.381 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.382 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:41.383 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.384 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:41.387 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.389 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.390 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.391 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.392 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:41.393 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.394 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.395 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.396 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:41.397 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.81:6812/847627118 conn(0x7f739c039a90 legacy=0x7f739c03bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 10 monclient: shutdown 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 >> v1:172.21.15.7:6793/0 conn(0x7f73b8137940 legacy=0x7f73b81cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73be0ec640 1 -- 172.21.15.7:0/4086935272 reap_dead start 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73be0ec640 1 -- 172.21.15.7:0/4086935272 reap_dead start 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 shutdown_connections 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 >> 172.21.15.7:0/4086935272 conn(0x7f73b80a9fa0 msgr2=0x7f73b80a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 shutdown_connections 2024-05-08T15:51:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.369+0000 7f73bfb4f640 1 -- 172.21.15.7:0/4086935272 wait complete. 2024-05-08T15:51:41.400 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:51:41.400 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon h 2024-05-08T15:51:41.400 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:51:41.401 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:41.402 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37185]: starting. 2024-05-08T15:51:41.402 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37185]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37083 2024-05-08T15:51:41.402 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37185]: shutting down. 2024-05-08T15:51:41.402 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:41.402 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:41.402 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:51:41.402 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.510+0000 7f7204225640 10 monclient: get_monmap_and_config 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.510+0000 7f7204225640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.510+0000 7f7204225640 10 monclient: monmap: 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:41.511222+0000 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:41.511222+0000 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.506 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.507 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.510+0000 7f7204225640 1 Processor -- start 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 1 -- start start 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient: init 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient: _add_conns ranks=[7,0,1,2,8,6,3,5,4] 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient(hunting): picked mon.noname-i con 0x7f71fc137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient(hunting): picked mon.noname-a con 0x7f71fc0a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient(hunting): picked mon.noname-f con 0x7f71fc0a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc0a2c70 con 0x7f71fc0a49a0 2024-05-08T15:51:41.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc0a2f50 con 0x7f71fc0a41b0 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc0a1670 con 0x7f71fc137940 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f7204225640 10 monclient(hunting): authenticate will time out at 1109.626312s 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f72017c0640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f71fc0a49a0 0x7f71fc13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50744/0 (socket says 172.21.15.7:50744) 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.511+0000 7f72017c0640 1 -- 172.21.15.7:0/1855394838 learned_addr learned my addr 172.21.15.7:0/1855394838 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196517482 0 0) 0x7f71fc0a2c70 con 0x7f71fc0a49a0 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): my global_id is 95013 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71e8003520 con 0x7f71fc0a49a0 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1658087076 0 0) 0x7f71fc0a2f50 con 0x7f71fc0a41b0 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): my global_id is 5707 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71fc0a2c70 con 0x7f71fc0a41b0 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777769082 0 0) 0x7f71fc0a1670 con 0x7f71fc137940 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient(hunting): my global_id is 114379 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71fc0a2f50 con 0x7f71fc137940 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4143405722 0 0) 0x7f71e8003520 con 0x7f71fc0a49a0 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 >> v1:172.21.15.81:6792/0 conn(0x7f71fc137940 legacy=0x7f71fc137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 >> v1:172.21.15.81:6789/0 conn(0x7f71fc0a41b0 legacy=0x7f71fc140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 monclient: found mon.noname-a 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc0a39f0 con 0x7f71fc0a49a0 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7204225640 5 monclient: authenticate success, global_id 95013 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f71ec0017b0 con 0x7f71fc0a49a0 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: dump: 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.519 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7204225640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f71ec001ad0 con 0x7f71fc0a49a0 2024-05-08T15:51:41.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 1 -- 172.21.15.7:0/1855394838 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f71ec004be0 con 0x7f71fc0a49a0 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7204225640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7204225640 4 set_mon_vals no callback set 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7200fbf640 10 monclient: dump: 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.521 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7204225640 10 monclient: shutdown 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7204225640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.512+0000 7f7204225640 1 -- 172.21.15.7:0/1855394838 >> v1:172.21.15.7:6789/0 conn(0x7f71fc0a49a0 legacy=0x7f71fc13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.513+0000 7f7204225640 1 -- 172.21.15.7:0/1855394838 shutdown_connections 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.513+0000 7f7204225640 1 -- 172.21.15.7:0/1855394838 >> 172.21.15.7:0/1855394838 conn(0x7f71fc0a9fa0 msgr2=0x7f71fc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.513+0000 7f7204225640 1 -- 172.21.15.7:0/1855394838 shutdown_connections 2024-05-08T15:51:41.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.513+0000 7f7204225640 1 -- 172.21.15.7:0/1855394838 wait complete. 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.513+0000 7f7204225640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.513+0000 7f7204225640 10 monclient: monmap: 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.523 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.513+0000 7f7204225640 1 Processor -- start 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 1 -- start start 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient: init 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient: _add_conns ranks=[7,2,5,6,1,4,0,8,3] 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient(hunting): picked mon.noname-i con 0x7f71fc0a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:51:41.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient(hunting): picked mon.noname-b con 0x7f71fc0a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient(hunting): picked mon.noname-h con 0x7f71fc137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc135a40 con 0x7f71fc0a49a0 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc1cda40 con 0x7f71fc137940 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71fc0a1f30 con 0x7f71fc0a41b0 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f7204225640 10 monclient(hunting): authenticate will time out at 1109.629111s 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f72017c0640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f71fc0a49a0 0x7f71fc1cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:60598/0 (socket says 172.21.15.7:60598) 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f72017c0640 1 -- 172.21.15.7:0/3050905526 learned_addr learned my addr 172.21.15.7:0/3050905526 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3782045005 0 0) 0x7f71fc135a40 con 0x7f71fc0a49a0 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f71f27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f71f27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f71f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f71f27fc640 10 monclient(hunting): my global_id is 95141 2024-05-08T15:51:41.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71e0003520 con 0x7f71fc0a49a0 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.514+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897482457 0 0) 0x7f71fc1cda40 con 0x7f71fc137940 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): my global_id is 5801 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71fc135a40 con 0x7f71fc137940 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740173416 0 0) 0x7f71fc0a1f30 con 0x7f71fc0a41b0 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): my global_id is 114388 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71fc1cda40 con 0x7f71fc0a41b0 2024-05-08T15:51:41.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 203304351 0 0) 0x7f71e0003520 con 0x7f71fc0a49a0 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f71fc0a1f30 con 0x7f71fc0a49a0 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f71ec004b20 con 0x7f71fc0a49a0 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): dump: 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.527 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 998700320 0 0) 0x7f71fc135a40 con 0x7f71fc137940 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f71e0003520 con 0x7f71fc137940 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f71f4001770 con 0x7f71fc137940 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient(hunting): dump: 2024-05-08T15:51:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 513754008 0 0) 0x7f71fc0a1f30 con 0x7f71fc0a49a0 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.81:6792/0 conn(0x7f71fc0a41b0 legacy=0x7f71fc1ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.81:6791/0 conn(0x7f71fc137940 legacy=0x7f71fc1d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 monclient: found mon.b 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc135760 con 0x7f71fc0a49a0 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f7204225640 5 monclient: authenticate success, global_id 95141 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f7204225640 10 monclient: _renew_subs 2024-05-08T15:51:41.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f7204225640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc134110 con 0x7f71fc0a49a0 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f7204225640 10 monclient: _renew_subs 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f7204225640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f71fc136140 con 0x7f71fc0a49a0 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f71ec0011b0 con 0x7f71fc0a49a0 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f71ec0017d0 con 0x7f71fc0a49a0 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f3fff640 4 set_mon_vals no callback set 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.515+0000 7f71f27fc640 10 monclient: dump: 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f7204225640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f71e40065c0 con 0x7f71e4002860 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f71ec010380 con 0x7f71fc0a49a0 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(200..200 src has 1..200) ==== 3506+0+0 (unknown 547594507 0 0) 0x7f71ec046c00 con 0x7f71fc0a49a0 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2780720620 0 0) 0x7f71e40065c0 con 0x7f71e4002860 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 10 monclient(hunting): my global_id is 5810 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71fc0a1f30 con 0x7f71e4002860 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f72027c2640 10 monclient: get_auth_request con 0x7f71e0039ec0 auth_method 0 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3900214096 0 0) 0x7f71fc0a1f30 con 0x7f71e4002860 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f71e40065c0 con 0x7f71e4002860 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f71f8003930 con 0x7f71e4002860 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.516+0000 7f71f27fc640 10 monclient: dump: 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.534 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.517+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1249367314 0 0) 0x7f71e40065c0 con 0x7f71e4002860 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.517+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71e400a150 con 0x7f71e4002860 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.517+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f71f8004f10 con 0x7f71e4002860 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.517+0000 7f71f27fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.517+0000 7f71f27fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.517+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.81:6791/0 conn(0x7f71e4002860 legacy=0x7f71e4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.517+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.81:6791/0 conn(0x7f71e4002860 legacy=0x7f71e4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.523+0000 7f7204225640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.523+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f71e4009e20 con 0x7f71e40090e0 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2513053488 0 0) 0x7f71e4009e20 con 0x7f71e40090e0 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient(hunting): my global_id is 5819 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f71e40065c0 con 0x7f71e40090e0 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3772533368 0 0) 0x7f71e40065c0 con 0x7f71e40090e0 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f71e4009e20 con 0x7f71e40090e0 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f71f80056d0 con 0x7f71e40090e0 2024-05-08T15:51:41.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient: dump: 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.538 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 12949321 0 0) 0x7f71e4009e20 con 0x7f71e40090e0 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f71e400c390 con 0x7f71e40090e0 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7671 (unknown 0 0 1768174202) 0x7f71f8005950 con 0x7f71e40090e0 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.81:6791/0 conn(0x7f71e40090e0 legacy=0x7f71e4009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.524+0000 7f71f27fc640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.81:6791/0 conn(0x7f71e40090e0 legacy=0x7f71e4009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.539 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 36, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 293830, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:41.540 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.541 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.542 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.543 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:41.544 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:41.544 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:41.544 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:41.544 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:41.544 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:41.544 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:41.544 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.545 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:41.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.550 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.551 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.552 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.553 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.554 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.555 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:41.556 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.557 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.558 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:51:41.559 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:41.560 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.81:6812/847627118 conn(0x7f71e0039ec0 legacy=0x7f71e003c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 10 monclient: shutdown 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 >> v1:172.21.15.7:6790/0 conn(0x7f71fc0a49a0 legacy=0x7f71fc1cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f72027c2640 1 -- 172.21.15.7:0/3050905526 reap_dead start 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f72027c2640 1 -- 172.21.15.7:0/3050905526 reap_dead start 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 shutdown_connections 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 >> 172.21.15.7:0/3050905526 conn(0x7f71fc0a9fa0 msgr2=0x7f71fc130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 shutdown_connections 2024-05-08T15:51:41.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.525+0000 7f7204225640 1 -- 172.21.15.7:0/3050905526 wait complete. 2024-05-08T15:51:41.563 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:51:41.563 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 7 for mon i 2024-05-08T15:51:41.563 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:51:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.671+0000 7f2a9988c640 10 monclient: get_monmap_and_config 2024-05-08T15:51:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.671+0000 7f2a9988c640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.671+0000 7f2a9988c640 10 monclient: monmap: 2024-05-08T15:51:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.667 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:51:41.672217+0000 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:51:41.672217+0000 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.668 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.669 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.669 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.669 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.669 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.669 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.671+0000 7f2a9988c640 1 Processor -- start 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.671+0000 7f2a9988c640 1 -- start start 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.671+0000 7f2a9988c640 10 monclient: init 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 10 monclient: _add_conns ranks=[3,4,0,6,1,8,2,7,5] 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 10 monclient(hunting): picked mon.noname-g con 0x7f2a940a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 10 monclient(hunting): picked mon.noname-c con 0x7f2a940a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 10 monclient(hunting): picked mon.noname-a con 0x7f2a9413e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:51:41.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a940a1660 con 0x7f2a9413e310 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a940a28f0 con 0x7f2a940a53a0 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a940a2bd0 con 0x7f2a940a5780 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a9988c640 10 monclient(hunting): authenticate will time out at 1109.787065s 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a8a7fc640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7f2a940a5780 legacy=0x7f2a9413cbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a937fe640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2a9413e310 0x7f2a94140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:50758/0 (socket says 172.21.15.7:50758) 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a937fe640 1 -- 172.21.15.7:0/1968469860 learned_addr learned my addr 172.21.15.7:0/1968469860 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 151957771 0 0) 0x7f2a940a1660 con 0x7f2a9413e310 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a927fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a927fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a927fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a927fc640 10 monclient(hunting): my global_id is 95022 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.672+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a80003520 con 0x7f2a9413e310 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559656341 0 0) 0x7f2a940a28f0 con 0x7f2a940a53a0 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient(hunting): my global_id is 76044 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a940a1660 con 0x7f2a940a53a0 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4149689930 0 0) 0x7f2a80003520 con 0x7f2a9413e310 2024-05-08T15:51:41.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 >> v1:172.21.15.7:6791/0 conn(0x7f2a940a5780 legacy=0x7f2a9413cbf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 >> v1:172.21.15.81:6790/0 conn(0x7f2a940a53a0 legacy=0x7f2a940a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 monclient: found mon.noname-a 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a940a6730 con 0x7f2a9413e310 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2a8c001b30 con 0x7f2a9413e310 2024-05-08T15:51:41.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 5 monclient: authenticate success, global_id 95022 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: dump: 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.682 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:51:41.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a8c001e50 con 0x7f2a9413e310 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 1 -- 172.21.15.7:0/1968469860 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2a8c004f60 con 0x7f2a9413e310 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 10 monclient: get_monmap_and_config success 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 4 set_mon_vals no callback set 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a927fc640 10 monclient: dump: 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.684 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 10 monclient: shutdown 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 1 -- 172.21.15.7:0/1968469860 >> v1:172.21.15.7:6789/0 conn(0x7f2a9413e310 legacy=0x7f2a94140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 1 -- 172.21.15.7:0/1968469860 shutdown_connections 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 1 -- 172.21.15.7:0/1968469860 >> 172.21.15.7:0/1968469860 conn(0x7f2a940a9fa0 msgr2=0x7f2a940aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 1 -- 172.21.15.7:0/1968469860 shutdown_connections 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.673+0000 7f2a9988c640 1 -- 172.21.15.7:0/1968469860 wait complete. 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.674+0000 7f2a9988c640 10 monclient: build_initial_monmap 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.674+0000 7f2a9988c640 10 monclient: monmap: 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:51:41.686 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:51:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:51:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:51:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:51:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:51:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:51:41.687 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:51:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:51:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.674+0000 7f2a9988c640 1 Processor -- start 2024-05-08T15:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.674+0000 7f2a9988c640 1 -- start start 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.674+0000 7f2a9988c640 10 monclient: init 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 10 monclient: _reopen_session rank -1 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 10 monclient: _add_conns ranks=[4,6,1,0,5,8,2,7,3] 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 10 monclient(hunting): picked mon.noname-c con 0x7f2a940a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 10 monclient(hunting): picked mon.noname-d con 0x7f2a940a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 10 monclient(hunting): picked mon.noname-f con 0x7f2a9413e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a941c9cd0 con 0x7f2a9413e310 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a92ffd640 1 -- >> v1:172.21.15.7:6791/0 conn(0x7f2a940a53a0 legacy=0x7f2a941c8eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6791/0 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a94135810 con 0x7f2a940a53a0 2024-05-08T15:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a8a7fc640 1 -- >> v1:172.21.15.7:6792/0 conn(0x7f2a940a5780 legacy=0x7f2a941c95c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.7:6792/0 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a94137260 con 0x7f2a940a5780 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 10 monclient(hunting): _renew_subs 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a9988c640 10 monclient(hunting): authenticate will time out at 1109.789998s 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a937fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2a9413e310 0x7f2a941d4790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:42594/0 (socket says 172.21.15.7:42594) 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a937fe640 1 -- 172.21.15.7:0/468231841 learned_addr learned my addr 172.21.15.7:0/468231841 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1356731191 0 0) 0x7f2a941c9cd0 con 0x7f2a9413e310 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a90ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a90ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a90ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a90ff9640 10 monclient(hunting): my global_id is 5716 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a74003520 con 0x7f2a9413e310 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.675+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1295211702 0 0) 0x7f2a74003520 con 0x7f2a9413e310 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a941c9cd0 con 0x7f2a9413e310 2024-05-08T15:51:41.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2a8c004ea0 con 0x7f2a9413e310 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient(hunting): dump: 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3649110283 0 0) 0x7f2a941c9cd0 con 0x7f2a9413e310 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.7:6792/0 conn(0x7f2a940a5780 legacy=0x7f2a941c95c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.7:6791/0 conn(0x7f2a940a53a0 legacy=0x7f2a941c8eb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 monclient: found mon.f 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a94137540 con 0x7f2a9413e310 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: _finish_auth 0 2024-05-08T15:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a9988c640 5 monclient: authenticate success, global_id 5716 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a9988c640 10 monclient: _renew_subs 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a9988c640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2a941365e0 con 0x7f2a9413e310 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a9988c640 10 monclient: _renew_subs 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a9988c640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2a941386c0 con 0x7f2a9413e310 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2a8c001e50 con 0x7f2a9413e310 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2a8c002000 con 0x7f2a9413e310 2024-05-08T15:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a91ffb640 4 set_mon_vals no callback set 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.676+0000 7f2a90ff9640 10 monclient: dump: 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a9988c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2a780065c0 con 0x7f2a78002860 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2a8c0109a0 con 0x7f2a9413e310 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(200..200 src has 1..200) ==== 3506+0+0 (unknown 547594507 0 0) 0x7f2a8c0471f0 con 0x7f2a9413e310 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a8a7fc640 10 monclient: get_auth_request con 0x7f2a74039970 auth_method 0 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 161244705 0 0) 0x7f2a780065c0 con 0x7f2a78002860 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 10 monclient(hunting): my global_id is 114397 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.677+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a941c9cd0 con 0x7f2a78002860 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1959803357 0 0) 0x7f2a941c9cd0 con 0x7f2a78002860 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a780065c0 con 0x7f2a78002860 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2a840018e0 con 0x7f2a78002860 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 10 monclient: dump: 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2306716054 0 0) 0x7f2a780065c0 con 0x7f2a78002860 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2a7800a150 con 0x7f2a78002860 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2a840057c0 con 0x7f2a78002860 2024-05-08T15:51:41.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:51:41.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:51:41.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.81:6792/0 conn(0x7f2a78002860 legacy=0x7f2a78004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.678+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.81:6792/0 conn(0x7f2a78002860 legacy=0x7f2a78004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a9988c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:51:41.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2a78009e40 con 0x7f2a78009100 2024-05-08T15:51:41.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2560888948 0 0) 0x7f2a78009e40 con 0x7f2a78009100 2024-05-08T15:51:41.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:51:41.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 10 monclient(hunting): my global_id is 114406 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a780065c0 con 0x7f2a78009100 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1081046961 0 0) 0x7f2a780065c0 con 0x7f2a78009100 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2a78009e40 con 0x7f2a78009100 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2a84003ba0 con 0x7f2a78009100 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 10 monclient: dump: 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:51:41.701 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:51:41.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2551821243 0 0) 0x7f2a78009e40 con 0x7f2a78009100 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.685+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2a7800c3d0 con 0x7f2a78009100 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7500 (unknown 0 0 598032284) 0x7f2a84005970 con 0x7f2a78009100 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a90ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a90ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.81:6792/0 conn(0x7f2a78009100 legacy=0x7f2a78009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a90ff9640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.81:6792/0 conn(0x7f2a78009100 legacy=0x7f2a78009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:41.703 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 36, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 36550, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:51:41.704 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.705 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:51:41.706 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:51:41.707 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:51:41.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.709 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.710 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.711 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.714 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.714 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:51:41.714 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.714 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.714 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.714 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.714 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.715 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.716 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.717 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.718 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.718 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.718 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.718 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:51:41.718 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:51:41.718 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.718 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.719 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.720 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.721 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:51:41.722 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.723 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:51:41.724 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.81:6812/847627118 conn(0x7f2a74039970 legacy=0x7f2a7403be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a9988c640 10 monclient: shutdown 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a9988c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 >> v1:172.21.15.81:6789/0 conn(0x7f2a9413e310 legacy=0x7f2a941d4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a937fe640 1 -- 172.21.15.7:0/468231841 reap_dead start 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a937fe640 1 -- 172.21.15.7:0/468231841 reap_dead start 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 shutdown_connections 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 >> 172.21.15.7:0/468231841 conn(0x7f2a940a9fa0 msgr2=0x7f2a94140c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:51:41.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.686+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 shutdown_connections 2024-05-08T15:51:41.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:51:41.687+0000 7f2a9988c640 1 -- 172.21.15.7:0/468231841 wait complete. 2024-05-08T15:51:41.727 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 7 2024-05-08T15:51:41.727 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-05-08T15:51:43.492 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37255]: starting. 2024-05-08T15:51:43.492 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37255]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37254 2024-05-08T15:51:43.492 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37255]: created object 0... 2024-05-08T15:51:43.492 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37255]: created object 25... 2024-05-08T15:51:43.492 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37255]: created object 49... 2024-05-08T15:51:43.492 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37255]: finishing. 2024-05-08T15:51:43.493 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37255]: shutting down. 2024-05-08T15:51:44.409 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37256]: starting. 2024-05-08T15:51:44.409 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37256]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37254 2024-05-08T15:51:44.409 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37256]: shutting down. 2024-05-08T15:51:46.523 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:46.523 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37292]: starting. 2024-05-08T15:51:46.524 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37292]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37254 2024-05-08T15:51:46.524 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37292]: created object 0... 2024-05-08T15:51:46.524 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37292]: created object 25... 2024-05-08T15:51:46.524 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37292]: created object 49... 2024-05-08T15:51:46.524 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37292]: finishing. 2024-05-08T15:51:46.524 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37292]: shutting down. 2024-05-08T15:51:46.534 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:46.534 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37294]: starting. 2024-05-08T15:51:46.534 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37294]: listing objects. 2024-05-08T15:51:46.535 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37294]: listed object 0... 2024-05-08T15:51:46.535 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37294]: listed object 25... 2024-05-08T15:51:46.535 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37294]: saw 50 objects 2024-05-08T15:51:46.535 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37294]: shutting down. 2024-05-08T15:51:47.434 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:47.434 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37293]: starting. 2024-05-08T15:51:47.434 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37293]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37254 2024-05-08T15:51:47.434 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37293]: shutting down. 2024-05-08T15:51:47.435 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:47.435 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:47.435 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:51:47.438 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:51:48.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37363]: starting. 2024-05-08T15:51:48.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37363]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37362 2024-05-08T15:51:48.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37363]: created object 0... 2024-05-08T15:51:48.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37363]: created object 25... 2024-05-08T15:51:48.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37363]: created object 49... 2024-05-08T15:51:48.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37363]: finishing. 2024-05-08T15:51:48.819 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37363]: shutting down. 2024-05-08T15:51:49.758 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37364]: starting. 2024-05-08T15:51:49.758 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37364]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37362 2024-05-08T15:51:49.758 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37364]: shutting down. 2024-05-08T15:51:51.852 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:51.852 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37399]: starting. 2024-05-08T15:51:51.852 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37399]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37362 2024-05-08T15:51:51.853 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37399]: created object 0... 2024-05-08T15:51:51.853 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37399]: created object 25... 2024-05-08T15:51:51.853 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37399]: created object 49... 2024-05-08T15:51:51.853 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37399]: finishing. 2024-05-08T15:51:51.853 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37399]: shutting down. 2024-05-08T15:51:51.864 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:51.864 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37401]: starting. 2024-05-08T15:51:51.864 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37401]: listing objects. 2024-05-08T15:51:51.864 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37401]: listed object 0... 2024-05-08T15:51:51.864 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37401]: listed object 25... 2024-05-08T15:51:51.864 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37401]: saw 50 objects 2024-05-08T15:51:51.864 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37401]: shutting down. 2024-05-08T15:51:52.772 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:52.772 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37400]: starting. 2024-05-08T15:51:52.772 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37400]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37362 2024-05-08T15:51:52.772 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37400]: shutting down. 2024-05-08T15:51:52.772 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:52.772 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:52.772 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:51:52.778 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:51:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37470]: starting. 2024-05-08T15:51:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37470]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37469 2024-05-08T15:51:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37470]: created object 0... 2024-05-08T15:51:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37470]: created object 25... 2024-05-08T15:51:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37470]: created object 49... 2024-05-08T15:51:54.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37470]: finishing. 2024-05-08T15:51:54.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37470]: shutting down. 2024-05-08T15:51:55.799 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37471]: starting. 2024-05-08T15:51:55.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37471]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37469 2024-05-08T15:51:55.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37471]: shutting down. 2024-05-08T15:51:57.838 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:57.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37506]: starting. 2024-05-08T15:51:57.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37506]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37469 2024-05-08T15:51:57.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37506]: created object 0... 2024-05-08T15:51:57.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37506]: created object 25... 2024-05-08T15:51:57.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37506]: created object 49... 2024-05-08T15:51:57.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37506]: finishing. 2024-05-08T15:51:57.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37506]: shutting down. 2024-05-08T15:51:57.850 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:57.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37508]: starting. 2024-05-08T15:51:57.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37508]: listing objects. 2024-05-08T15:51:57.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37508]: listed object 0... 2024-05-08T15:51:57.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37508]: listed object 25... 2024-05-08T15:51:57.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37508]: saw 50 objects 2024-05-08T15:51:57.850 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37508]: shutting down. 2024-05-08T15:51:58.743 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:58.743 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37507]: starting. 2024-05-08T15:51:58.743 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37507]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37469 2024-05-08T15:51:58.743 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37507]: shutting down. 2024-05-08T15:51:58.744 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:58.744 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:51:58.744 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:51:58.747 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:00.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37578]: starting. 2024-05-08T15:52:00.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37578]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37577 2024-05-08T15:52:00.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37578]: created object 0... 2024-05-08T15:52:00.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37578]: created object 25... 2024-05-08T15:52:00.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37578]: created object 49... 2024-05-08T15:52:00.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37578]: finishing. 2024-05-08T15:52:00.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37578]: shutting down. 2024-05-08T15:52:01.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37579]: starting. 2024-05-08T15:52:01.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37579]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37577 2024-05-08T15:52:01.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37579]: shutting down. 2024-05-08T15:52:03.874 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:03.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37614]: starting. 2024-05-08T15:52:03.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37614]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37577 2024-05-08T15:52:03.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37614]: created object 0... 2024-05-08T15:52:03.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37614]: created object 25... 2024-05-08T15:52:03.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37614]: created object 49... 2024-05-08T15:52:03.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37614]: finishing. 2024-05-08T15:52:03.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37614]: shutting down. 2024-05-08T15:52:03.885 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:03.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37616]: starting. 2024-05-08T15:52:03.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37616]: listing objects. 2024-05-08T15:52:03.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37616]: listed object 0... 2024-05-08T15:52:03.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37616]: listed object 25... 2024-05-08T15:52:03.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37616]: saw 50 objects 2024-05-08T15:52:03.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37616]: shutting down. 2024-05-08T15:52:04.779 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:04.780 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37615]: starting. 2024-05-08T15:52:04.780 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37615]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37577 2024-05-08T15:52:04.780 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37615]: shutting down. 2024-05-08T15:52:04.781 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:04.781 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:04.781 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:04.784 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:06.881 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37685]: starting. 2024-05-08T15:52:06.881 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37685]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37684 2024-05-08T15:52:06.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37685]: created object 0... 2024-05-08T15:52:06.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37685]: created object 25... 2024-05-08T15:52:06.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37685]: created object 49... 2024-05-08T15:52:06.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37685]: finishing. 2024-05-08T15:52:06.882 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37685]: shutting down. 2024-05-08T15:52:07.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37686]: starting. 2024-05-08T15:52:07.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37686]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37684 2024-05-08T15:52:07.742 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37686]: shutting down. 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37721]: starting. 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37721]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37684 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37721]: created object 0... 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37721]: created object 25... 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37721]: created object 49... 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37721]: finishing. 2024-05-08T15:52:09.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37721]: shutting down. 2024-05-08T15:52:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37723]: starting. 2024-05-08T15:52:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37723]: listing objects. 2024-05-08T15:52:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37723]: listed object 0... 2024-05-08T15:52:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37723]: listed object 25... 2024-05-08T15:52:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37723]: saw 50 objects 2024-05-08T15:52:09.851 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37723]: shutting down. 2024-05-08T15:52:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37722]: starting. 2024-05-08T15:52:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37722]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37684 2024-05-08T15:52:10.746 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37722]: shutting down. 2024-05-08T15:52:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:10.747 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:10.750 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:12.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37792]: starting. 2024-05-08T15:52:12.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37792]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37791 2024-05-08T15:52:12.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37792]: created object 0... 2024-05-08T15:52:12.859 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37792]: created object 25... 2024-05-08T15:52:12.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37792]: created object 49... 2024-05-08T15:52:12.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37792]: finishing. 2024-05-08T15:52:12.860 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37792]: shutting down. 2024-05-08T15:52:13.781 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37793]: starting. 2024-05-08T15:52:13.781 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37793]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37791 2024-05-08T15:52:13.781 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37793]: shutting down. 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37828]: starting. 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37828]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37791 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37828]: created object 0... 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37828]: created object 25... 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37828]: created object 49... 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37828]: finishing. 2024-05-08T15:52:15.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37828]: shutting down. 2024-05-08T15:52:15.894 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:15.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37830]: starting. 2024-05-08T15:52:15.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37830]: listing objects. 2024-05-08T15:52:15.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37830]: listed object 0... 2024-05-08T15:52:15.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37830]: listed object 25... 2024-05-08T15:52:15.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37830]: saw 50 objects 2024-05-08T15:52:15.894 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37830]: shutting down. 2024-05-08T15:52:16.797 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:16.797 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37829]: starting. 2024-05-08T15:52:16.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37829]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37791 2024-05-08T15:52:16.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37829]: shutting down. 2024-05-08T15:52:16.799 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:16.799 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:16.799 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:16.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:18.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37899]: starting. 2024-05-08T15:52:18.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37899]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37898 2024-05-08T15:52:18.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37899]: created object 0... 2024-05-08T15:52:18.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37899]: created object 25... 2024-05-08T15:52:18.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37899]: created object 49... 2024-05-08T15:52:18.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37899]: finishing. 2024-05-08T15:52:18.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[37899]: shutting down. 2024-05-08T15:52:19.755 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37900]: starting. 2024-05-08T15:52:19.755 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37900]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37898 2024-05-08T15:52:19.755 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[37900]: shutting down. 2024-05-08T15:52:21.860 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:21.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37935]: starting. 2024-05-08T15:52:21.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37935]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-37898 2024-05-08T15:52:21.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37935]: created object 0... 2024-05-08T15:52:21.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37935]: created object 25... 2024-05-08T15:52:21.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37935]: created object 49... 2024-05-08T15:52:21.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37935]: finishing. 2024-05-08T15:52:21.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[37935]: shutting down. 2024-05-08T15:52:21.871 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37937]: starting. 2024-05-08T15:52:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37937]: listing objects. 2024-05-08T15:52:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37937]: listed object 0... 2024-05-08T15:52:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37937]: listed object 25... 2024-05-08T15:52:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37937]: saw 50 objects 2024-05-08T15:52:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[37937]: shutting down. 2024-05-08T15:52:22.775 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:22.775 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37936]: starting. 2024-05-08T15:52:22.776 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37936]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-37898 2024-05-08T15:52:22.776 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[37936]: shutting down. 2024-05-08T15:52:22.777 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:22.777 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:22.777 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:22.779 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38006]: starting. 2024-05-08T15:52:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38006]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38005 2024-05-08T15:52:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38006]: created object 0... 2024-05-08T15:52:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38006]: created object 25... 2024-05-08T15:52:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38006]: created object 49... 2024-05-08T15:52:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38006]: finishing. 2024-05-08T15:52:24.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38006]: shutting down. 2024-05-08T15:52:25.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38007]: starting. 2024-05-08T15:52:25.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38007]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38005 2024-05-08T15:52:25.818 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38007]: shutting down. 2024-05-08T15:52:27.832 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:27.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38042]: starting. 2024-05-08T15:52:27.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38042]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38005 2024-05-08T15:52:27.832 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38042]: created object 0... 2024-05-08T15:52:27.833 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38042]: created object 25... 2024-05-08T15:52:27.833 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38042]: created object 49... 2024-05-08T15:52:27.833 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38042]: finishing. 2024-05-08T15:52:27.833 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38042]: shutting down. 2024-05-08T15:52:27.843 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:27.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38044]: starting. 2024-05-08T15:52:27.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38044]: listing objects. 2024-05-08T15:52:27.843 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38044]: listed object 0... 2024-05-08T15:52:27.844 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38044]: listed object 25... 2024-05-08T15:52:27.844 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38044]: saw 50 objects 2024-05-08T15:52:27.844 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38044]: shutting down. 2024-05-08T15:52:28.754 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:28.754 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38043]: starting. 2024-05-08T15:52:28.754 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38043]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38005 2024-05-08T15:52:28.754 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38043]: shutting down. 2024-05-08T15:52:28.755 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:28.755 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:28.755 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:28.758 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:30.854 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38113]: starting. 2024-05-08T15:52:30.854 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38113]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38112 2024-05-08T15:52:30.854 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38113]: created object 0... 2024-05-08T15:52:30.855 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38113]: created object 25... 2024-05-08T15:52:30.855 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38113]: created object 49... 2024-05-08T15:52:30.855 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38113]: finishing. 2024-05-08T15:52:30.855 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38113]: shutting down. 2024-05-08T15:52:31.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38114]: starting. 2024-05-08T15:52:31.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38114]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38112 2024-05-08T15:52:31.762 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38114]: shutting down. 2024-05-08T15:52:33.867 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:33.868 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38149]: starting. 2024-05-08T15:52:33.868 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38149]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38112 2024-05-08T15:52:33.868 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38149]: created object 0... 2024-05-08T15:52:33.868 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38149]: created object 25... 2024-05-08T15:52:33.868 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38149]: created object 49... 2024-05-08T15:52:33.868 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38149]: finishing. 2024-05-08T15:52:33.868 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38149]: shutting down. 2024-05-08T15:52:33.879 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:33.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38151]: starting. 2024-05-08T15:52:33.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38151]: listing objects. 2024-05-08T15:52:33.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38151]: listed object 0... 2024-05-08T15:52:33.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38151]: listed object 25... 2024-05-08T15:52:33.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38151]: saw 50 objects 2024-05-08T15:52:33.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38151]: shutting down. 2024-05-08T15:52:34.791 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:34.791 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38150]: starting. 2024-05-08T15:52:34.791 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38150]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38112 2024-05-08T15:52:34.792 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38150]: shutting down. 2024-05-08T15:52:34.793 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:34.793 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:34.793 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:34.796 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:36.906 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38220]: starting. 2024-05-08T15:52:36.907 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38220]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38219 2024-05-08T15:52:36.907 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38220]: created object 0... 2024-05-08T15:52:36.907 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38220]: created object 25... 2024-05-08T15:52:36.907 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38220]: created object 49... 2024-05-08T15:52:36.907 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38220]: finishing. 2024-05-08T15:52:36.907 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38220]: shutting down. 2024-05-08T15:52:37.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38221]: starting. 2024-05-08T15:52:37.757 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38221]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38219 2024-05-08T15:52:37.757 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38221]: shutting down. 2024-05-08T15:52:39.860 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:39.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38256]: starting. 2024-05-08T15:52:39.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38256]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38219 2024-05-08T15:52:39.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38256]: created object 0... 2024-05-08T15:52:39.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38256]: created object 25... 2024-05-08T15:52:39.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38256]: created object 49... 2024-05-08T15:52:39.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38256]: finishing. 2024-05-08T15:52:39.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38256]: shutting down. 2024-05-08T15:52:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38258]: starting. 2024-05-08T15:52:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38258]: listing objects. 2024-05-08T15:52:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38258]: listed object 0... 2024-05-08T15:52:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38258]: listed object 25... 2024-05-08T15:52:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38258]: saw 50 objects 2024-05-08T15:52:39.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38258]: shutting down. 2024-05-08T15:52:40.773 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:40.773 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38257]: starting. 2024-05-08T15:52:40.773 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38257]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38219 2024-05-08T15:52:40.773 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38257]: shutting down. 2024-05-08T15:52:40.774 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:40.774 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:40.774 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:40.779 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:42.876 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38329]: starting. 2024-05-08T15:52:42.877 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38329]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38328 2024-05-08T15:52:42.877 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38329]: created object 0... 2024-05-08T15:52:42.877 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38329]: created object 25... 2024-05-08T15:52:42.877 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38329]: created object 49... 2024-05-08T15:52:42.877 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38329]: finishing. 2024-05-08T15:52:42.877 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38329]: shutting down. 2024-05-08T15:52:43.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38330]: starting. 2024-05-08T15:52:43.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38330]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38328 2024-05-08T15:52:43.806 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38330]: shutting down. 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38366]: starting. 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38366]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38328 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38366]: created object 0... 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38366]: created object 25... 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38366]: created object 49... 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38366]: finishing. 2024-05-08T15:52:45.903 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38366]: shutting down. 2024-05-08T15:52:45.913 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:45.913 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38368]: starting. 2024-05-08T15:52:45.913 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38368]: listing objects. 2024-05-08T15:52:45.913 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38368]: listed object 0... 2024-05-08T15:52:45.913 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38368]: listed object 25... 2024-05-08T15:52:45.914 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38368]: saw 50 objects 2024-05-08T15:52:45.914 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38368]: shutting down. 2024-05-08T15:52:46.829 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:46.829 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38367]: starting. 2024-05-08T15:52:46.829 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38367]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38328 2024-05-08T15:52:46.829 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38367]: shutting down. 2024-05-08T15:52:46.830 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:46.831 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:46.831 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:46.833 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:48.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38437]: starting. 2024-05-08T15:52:48.841 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38437]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38436 2024-05-08T15:52:48.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38437]: created object 0... 2024-05-08T15:52:48.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38437]: created object 25... 2024-05-08T15:52:48.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38437]: created object 49... 2024-05-08T15:52:48.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38437]: finishing. 2024-05-08T15:52:48.842 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38437]: shutting down. 2024-05-08T15:52:49.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38438]: starting. 2024-05-08T15:52:49.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38438]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38436 2024-05-08T15:52:49.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38438]: shutting down. 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38473]: starting. 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38473]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38436 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38473]: created object 0... 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38473]: created object 25... 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38473]: created object 49... 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38473]: finishing. 2024-05-08T15:52:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38473]: shutting down. 2024-05-08T15:52:51.872 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:51.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38475]: starting. 2024-05-08T15:52:51.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38475]: listing objects. 2024-05-08T15:52:51.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38475]: listed object 0... 2024-05-08T15:52:51.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38475]: listed object 25... 2024-05-08T15:52:51.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38475]: saw 50 objects 2024-05-08T15:52:51.872 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38475]: shutting down. 2024-05-08T15:52:52.778 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:52.778 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38474]: starting. 2024-05-08T15:52:52.778 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38474]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38436 2024-05-08T15:52:52.778 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38474]: shutting down. 2024-05-08T15:52:52.780 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:52.780 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:52.780 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:52.782 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:52:54.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38546]: starting. 2024-05-08T15:52:54.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38546]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38545 2024-05-08T15:52:54.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38546]: created object 0... 2024-05-08T15:52:54.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38546]: created object 25... 2024-05-08T15:52:54.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38546]: created object 49... 2024-05-08T15:52:54.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38546]: finishing. 2024-05-08T15:52:54.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38546]: shutting down. 2024-05-08T15:52:55.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38547]: starting. 2024-05-08T15:52:55.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38547]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38545 2024-05-08T15:52:55.798 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38547]: shutting down. 2024-05-08T15:52:57.835 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:57.835 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38583]: starting. 2024-05-08T15:52:57.835 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38583]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38545 2024-05-08T15:52:57.835 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38583]: created object 0... 2024-05-08T15:52:57.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38583]: created object 25... 2024-05-08T15:52:57.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38583]: created object 49... 2024-05-08T15:52:57.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38583]: finishing. 2024-05-08T15:52:57.836 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38583]: shutting down. 2024-05-08T15:52:57.846 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:57.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38585]: starting. 2024-05-08T15:52:57.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38585]: listing objects. 2024-05-08T15:52:57.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38585]: listed object 0... 2024-05-08T15:52:57.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38585]: listed object 25... 2024-05-08T15:52:57.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38585]: saw 50 objects 2024-05-08T15:52:57.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38585]: shutting down. 2024-05-08T15:52:58.756 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:58.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38584]: starting. 2024-05-08T15:52:58.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38584]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38545 2024-05-08T15:52:58.756 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38584]: shutting down. 2024-05-08T15:52:58.757 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:58.757 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:52:58.757 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:52:58.760 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:53:00.858 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38655]: starting. 2024-05-08T15:53:00.858 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38655]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38654 2024-05-08T15:53:00.858 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38655]: created object 0... 2024-05-08T15:53:00.858 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38655]: created object 25... 2024-05-08T15:53:00.858 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38655]: created object 49... 2024-05-08T15:53:00.858 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38655]: finishing. 2024-05-08T15:53:00.858 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38655]: shutting down. 2024-05-08T15:53:01.771 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38656]: starting. 2024-05-08T15:53:01.771 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38656]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38654 2024-05-08T15:53:01.771 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38656]: shutting down. 2024-05-08T15:53:03.952 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:03.952 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38691]: starting. 2024-05-08T15:53:03.952 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38691]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38654 2024-05-08T15:53:03.953 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38691]: created object 0... 2024-05-08T15:53:03.953 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38691]: created object 25... 2024-05-08T15:53:03.953 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38691]: created object 49... 2024-05-08T15:53:03.953 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38691]: finishing. 2024-05-08T15:53:03.953 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38691]: shutting down. 2024-05-08T15:53:03.968 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:03.968 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38693]: starting. 2024-05-08T15:53:03.968 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38693]: listing objects. 2024-05-08T15:53:03.968 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38693]: listed object 0... 2024-05-08T15:53:03.968 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38693]: listed object 25... 2024-05-08T15:53:03.968 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38693]: saw 50 objects 2024-05-08T15:53:03.968 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38693]: shutting down. 2024-05-08T15:53:04.794 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:04.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38692]: starting. 2024-05-08T15:53:04.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38692]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38654 2024-05-08T15:53:04.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38692]: shutting down. 2024-05-08T15:53:04.795 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:04.795 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:04.796 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:53:04.798 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:53:06.883 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38762]: starting. 2024-05-08T15:53:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38762]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38761 2024-05-08T15:53:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38762]: created object 0... 2024-05-08T15:53:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38762]: created object 25... 2024-05-08T15:53:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38762]: created object 49... 2024-05-08T15:53:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38762]: finishing. 2024-05-08T15:53:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38762]: shutting down. 2024-05-08T15:53:07.760 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38763]: starting. 2024-05-08T15:53:07.760 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38763]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38761 2024-05-08T15:53:07.760 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38763]: shutting down. 2024-05-08T15:53:09.866 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:09.866 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38798]: starting. 2024-05-08T15:53:09.866 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38798]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38761 2024-05-08T15:53:09.866 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38798]: created object 0... 2024-05-08T15:53:09.866 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38798]: created object 25... 2024-05-08T15:53:09.867 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38798]: created object 49... 2024-05-08T15:53:09.867 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38798]: finishing. 2024-05-08T15:53:09.867 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38798]: shutting down. 2024-05-08T15:53:09.878 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:09.878 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38800]: starting. 2024-05-08T15:53:09.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38800]: listing objects. 2024-05-08T15:53:09.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38800]: listed object 0... 2024-05-08T15:53:09.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38800]: listed object 25... 2024-05-08T15:53:09.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38800]: saw 50 objects 2024-05-08T15:53:09.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38800]: shutting down. 2024-05-08T15:53:10.771 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:10.771 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38799]: starting. 2024-05-08T15:53:10.771 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38799]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38761 2024-05-08T15:53:10.771 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38799]: shutting down. 2024-05-08T15:53:10.772 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:10.773 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:10.773 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:53:10.775 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:53:11.727 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-05-08T15:53:11.728 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-05-08T15:53:11.728 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T15:53:11.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T15:53:11.731 INFO:tasks.ceph.mon.c:Started 2024-05-08T15:53:11.731 INFO:tasks.mon_thrash.mon_thrasher:killing mon.d 2024-05-08T15:53:11.731 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.d 2024-05-08T15:53:11.731 INFO:tasks.ceph.mon.d:Restarting daemon 2024-05-08T15:53:11.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i d 2024-05-08T15:53:11.733 INFO:tasks.ceph.mon.d:Started 2024-05-08T15:53:11.734 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-05-08T15:53:11.734 INFO:tasks.ceph.mon.a:Sent signal 19 2024-05-08T15:53:11.734 INFO:tasks.mon_thrash:Sending STOP to mon i 2024-05-08T15:53:11.734 INFO:tasks.ceph.mon.i:Sent signal 19 2024-05-08T15:53:11.734 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-05-08T15:53:26.735 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-05-08T15:53:26.735 INFO:tasks.ceph.mon.a:Sent signal 18 2024-05-08T15:53:26.735 INFO:tasks.mon_thrash:Sending CONT to mon i 2024-05-08T15:53:26.736 INFO:tasks.ceph.mon.i:Sent signal 18 2024-05-08T15:53:26.736 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:53:26.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:53:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.865+0000 7f5a28a6d640 10 monclient: get_monmap_and_config 2024-05-08T15:53:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.865+0000 7f5a28a6d640 10 monclient: build_initial_monmap 2024-05-08T15:53:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.865+0000 7f5a28a6d640 10 monclient: monmap: 2024-05-08T15:53:26.861 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:26.866665+0000 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:26.866665+0000 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:26.863 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.866+0000 7f5a28a6d640 1 Processor -- start 2024-05-08T15:53:26.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.866+0000 7f5a28a6d640 1 -- start start 2024-05-08T15:53:26.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.866+0000 7f5a28a6d640 10 monclient: init 2024-05-08T15:53:26.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.866+0000 7f5a28a6d640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.866+0000 7f5a28a6d640 10 monclient: _add_conns ranks=[1,5,2,8,6,0,3,4,7] 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 10 monclient(hunting): picked mon.noname-f con 0x7f5a240a1580 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 10 monclient(hunting): picked mon.noname-h con 0x7f5a240a2070 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 10 monclient(hunting): picked mon.noname-b con 0x7f5a24072690 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a24072aa0 con 0x7f5a240a1580 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a24070160 con 0x7f5a24072690 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a24071410 con 0x7f5a240a2070 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 10 monclient(hunting): authenticate will time out at 1214.981724s 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a22d9d640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f5a24072690 0x7f5a2413d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51876/0 (socket says 172.21.15.7:51876) 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a22d9d640 1 -- 172.21.15.7:0/853979569 learned_addr learned my addr 172.21.15.7:0/853979569 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3662407471 0 0) 0x7f5a24070160 con 0x7f5a24072690 2024-05-08T15:53:26.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): my global_id is 95591 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a0c003520 con 0x7f5a24072690 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3503611975 0 0) 0x7f5a24071410 con 0x7f5a240a2070 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): my global_id is 6278 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a24070160 con 0x7f5a240a2070 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726640860 0 0) 0x7f5a24072aa0 con 0x7f5a240a1580 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:26.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient(hunting): my global_id is 6076 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a24071410 con 0x7f5a240a1580 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3567847011 0 0) 0x7f5a0c003520 con 0x7f5a24072690 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 >> v1:172.21.15.81:6791/0 conn(0x7f5a240a2070 legacy=0x7f5a24139a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 >> v1:172.21.15.81:6789/0 conn(0x7f5a240a1580 legacy=0x7f5a240a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: _finish_hunting 0 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 monclient: found mon.noname-b 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2406fcd0 con 0x7f5a24072690 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: _finish_auth 0 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: _check_auth_tickets 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 5 monclient: authenticate success, global_id 95591 2024-05-08T15:53:26.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5a1c001b30 con 0x7f5a24072690 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: dump: 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:26.876 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5a1c001de0 con 0x7f5a24072690 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 1 -- 172.21.15.7:0/853979569 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5a1c004f60 con 0x7f5a24072690 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a28a6d640 4 set_mon_vals no callback set 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:26.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.867+0000 7f5a2159a640 10 monclient: dump: 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:26.878 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.868+0000 7f5a28a6d640 10 monclient: shutdown 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.868+0000 7f5a28a6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.868+0000 7f5a28a6d640 1 -- 172.21.15.7:0/853979569 >> v1:172.21.15.7:6790/0 conn(0x7f5a24072690 legacy=0x7f5a2413d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.868+0000 7f5a28a6d640 1 -- 172.21.15.7:0/853979569 shutdown_connections 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.868+0000 7f5a28a6d640 1 -- 172.21.15.7:0/853979569 >> 172.21.15.7:0/853979569 conn(0x7f5a240a6280 msgr2=0x7f5a240a2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.868+0000 7f5a28a6d640 1 -- 172.21.15.7:0/853979569 shutdown_connections 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.868+0000 7f5a28a6d640 1 -- 172.21.15.7:0/853979569 wait complete. 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.869+0000 7f5a28a6d640 10 monclient: build_initial_monmap 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.869+0000 7f5a28a6d640 10 monclient: monmap: 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:26.879 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.869+0000 7f5a28a6d640 1 Processor -- start 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.869+0000 7f5a28a6d640 1 -- start start 2024-05-08T15:53:26.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.869+0000 7f5a28a6d640 10 monclient: init 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 10 monclient: _add_conns ranks=[8,7,6,4,0,5,3,2,1] 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 10 monclient(hunting): picked mon.noname-e con 0x7f5a24072690 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 10 monclient(hunting): picked mon.noname-i con 0x7f5a240a1580 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 10 monclient(hunting): picked mon.noname-d con 0x7f5a240a2070 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2409af00 con 0x7f5a240a2070 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2413b5c0 con 0x7f5a240a1580 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a2413bc70 con 0x7f5a24072690 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a28a6d640 10 monclient(hunting): authenticate will time out at 1214.984834s 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a2259c640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f5a24072690 0x7f5a2412b550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35514/0 (socket says 172.21.15.7:35514) 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a2259c640 1 -- 172.21.15.7:0/1822330318 learned_addr learned my addr 172.21.15.7:0/1822330318 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313787844 0 0) 0x7f5a2413bc70 con 0x7f5a24072690 2024-05-08T15:53:26.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): my global_id is 95597 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a04003520 con 0x7f5a24072690 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2971074431 0 0) 0x7f5a2406fcd0 con 0x7f5a240a2070 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): my global_id is 124107 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a2413bc70 con 0x7f5a240a2070 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2691584069 0 0) 0x7f5a2413b5c0 con 0x7f5a240a1580 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:26.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): my global_id is 114856 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5a2406fcd0 con 0x7f5a240a1580 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2249610175 0 0) 0x7f5a04003520 con 0x7f5a24072690 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a2413b5c0 con 0x7f5a24072690 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5a14001b30 con 0x7f5a24072690 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:26.883 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1649528652 0 0) 0x7f5a2413bc70 con 0x7f5a240a2070 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a04003520 con 0x7f5a240a2070 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5a1c004ea0 con 0x7f5a240a2070 2024-05-08T15:53:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:26.885 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1435086833 0 0) 0x7f5a2406fcd0 con 0x7f5a240a1580 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5a2413bc70 con 0x7f5a240a1580 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5a18001890 con 0x7f5a240a1580 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.870+0000 7f5a12ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.886 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1202499289 0 0) 0x7f5a2413b5c0 con 0x7f5a24072690 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 >> v1:172.21.15.81:6792/0 conn(0x7f5a240a1580 legacy=0x7f5a241d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:26.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 >> v1:172.21.15.7:6792/0 conn(0x7f5a240a2070 legacy=0x7f5a241d5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 monclient: found mon.e 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2413d120 con 0x7f5a24072690 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a28a6d640 5 monclient: authenticate success, global_id 95597 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a28a6d640 10 monclient: _renew_subs 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a28a6d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2413ce40 con 0x7f5a24072690 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a28a6d640 10 monclient: _renew_subs 2024-05-08T15:53:26.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a28a6d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f5a2413b9c0 con 0x7f5a24072690 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5a140038c0 con 0x7f5a24072690 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5a14003db0 con 0x7f5a24072690 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a20d99640 4 set_mon_vals no callback set 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 10 monclient: dump: 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:26.889 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.871+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f5a14003f50 con 0x7f5a24072690 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.872+0000 7f5a28a6d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.872+0000 7f5a28a6d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.872+0000 7f5a28a6d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:26.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.872+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a08005080 con 0x7f5a24072690 2024-05-08T15:53:26.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.872+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(293..293 src has 1..293) ==== 3898+0+0 (unknown 1971243629 0 0) 0x7f5a14046f00 con 0x7f5a24072690 2024-05-08T15:53:26.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.872+0000 7f5a21d9b640 10 monclient: get_auth_request con 0x7f5a0403a500 auth_method 0 2024-05-08T15:53:26.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.877+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f5a14015f80 con 0x7f5a24072690 2024-05-08T15:53:26.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.877+0000 7f5a12ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:26.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:26.877+0000 7f5a12ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.133+0000 7f5a28a6d640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:53:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.133+0000 7f5a28a6d640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:53:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.133+0000 7f5a28a6d640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:27.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.133+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5a08005370 con 0x7f5a24072690 2024-05-08T15:53:27.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.133+0000 7f5a12ffd640 1 -- 172.21.15.7:0/1822330318 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 2276063604) 0x7f5a14015880 con 0x7f5a24072690 2024-05-08T15:53:27.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.133+0000 7f5a12ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:53:27.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.133+0000 7f5a12ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:27.130 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":50,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:53:27.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 >> v1:172.21.15.81:6812/847627118 conn(0x7f5a0403a500 legacy=0x7f5a0403c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 10 monclient: shutdown 2024-05-08T15:53:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 >> v1:172.21.15.7:6793/0 conn(0x7f5a24072690 legacy=0x7f5a2412b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 shutdown_connections 2024-05-08T15:53:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 >> 172.21.15.7:0/1822330318 conn(0x7f5a240a6280 msgr2=0x7f5a240a4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 shutdown_connections 2024-05-08T15:53:27.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.136+0000 7f5a28a6d640 1 -- 172.21.15.7:0/1822330318 wait complete. 2024-05-08T15:53:27.143 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:53:27.143 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:53:27.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:53:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.252+0000 7fe9f3932640 10 monclient: get_monmap_and_config 2024-05-08T15:53:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.252+0000 7fe9f3932640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.252+0000 7fe9f3932640 10 monclient: monmap: 2024-05-08T15:53:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.264 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:27.253479+0000 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:27.253479+0000 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 1 Processor -- start 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 1 -- start start 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient: init 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient: _add_conns ranks=[0,3,8,5,7,2,4,6,1] 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient(hunting): picked mon.noname-a con 0x7fe9ec137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient(hunting): picked mon.noname-g con 0x7fe9ec0a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient(hunting): picked mon.noname-e con 0x7fe9ec0a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec0a2c70 con 0x7fe9ec137940 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec0a2f50 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec0a1670 con 0x7fe9ec0a41b0 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f3932640 10 monclient(hunting): authenticate will time out at 1215.368398s 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f1ecf640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fe9ec0a41b0 0x7fe9ec140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35522/0 (socket says 172.21.15.7:35522) 2024-05-08T15:53:27.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f16ce640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe9ec137940 0x7fe9ec137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39042/0 (socket says 172.21.15.7:39042) 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.253+0000 7fe9f1ecf640 1 -- 172.21.15.7:0/4195822820 learned_addr learned my addr 172.21.15.7:0/4195822820 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104992013 0 0) 0x7fe9ec0a2c70 con 0x7fe9ec137940 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): my global_id is 95526 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9d4003520 con 0x7fe9ec137940 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3894577936 0 0) 0x7fe9ec0a1670 con 0x7fe9ec0a41b0 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): my global_id is 95606 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9ec0a2c70 con 0x7fe9ec0a41b0 2024-05-08T15:53:27.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687432069 0 0) 0x7fe9ec0a2f50 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient(hunting): my global_id is 76413 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9ec0a1670 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3375568888 0 0) 0x7fe9d4003520 con 0x7fe9ec137940 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 >> v1:172.21.15.7:6793/0 conn(0x7fe9ec0a41b0 legacy=0x7fe9ec140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 >> v1:172.21.15.81:6790/0 conn(0x7fe9ec0a49a0 legacy=0x7fe9ec13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 monclient: found mon.noname-a 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:53:27.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec0a39f0 con 0x7fe9ec137940 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe9dc001b30 con 0x7fe9ec137940 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9f3932640 5 monclient: authenticate success, global_id 95526 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9f3932640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: dump: 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.269 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9f3932640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe9dc001e50 con 0x7fe9ec137940 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 1 -- 172.21.15.7:0/4195822820 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe9dc0020c0 con 0x7fe9ec137940 2024-05-08T15:53:27.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9f3932640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9f3932640 4 set_mon_vals no callback set 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.254+0000 7fe9ebfff640 10 monclient: dump: 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.271 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 10 monclient: shutdown 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 1 -- 172.21.15.7:0/4195822820 >> v1:172.21.15.7:6789/0 conn(0x7fe9ec137940 legacy=0x7fe9ec137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 1 -- 172.21.15.7:0/4195822820 shutdown_connections 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 1 -- 172.21.15.7:0/4195822820 >> 172.21.15.7:0/4195822820 conn(0x7fe9ec0a9fa0 msgr2=0x7fe9ec0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 1 -- 172.21.15.7:0/4195822820 shutdown_connections 2024-05-08T15:53:27.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 1 -- 172.21.15.7:0/4195822820 wait complete. 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.255+0000 7fe9f3932640 10 monclient: monmap: 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.273 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 1 Processor -- start 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 1 -- start start 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient: init 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient: _add_conns ranks=[1,4,5,7,8,6,3,2,0] 2024-05-08T15:53:27.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient(hunting): picked mon.noname-f con 0x7fe9ec0a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient(hunting): picked mon.noname-c con 0x7fe9ec0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient(hunting): picked mon.noname-h con 0x7fe9ec137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec135530 con 0x7fe9ec0a41b0 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec136900 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ec1d7ab0 con 0x7fe9ec137940 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f3932640 10 monclient(hunting): authenticate will time out at 1215.371197s 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f0ecd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fe9ec0a49a0 0x7fe9ec1d0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51802/0 (socket says 172.21.15.7:51802) 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9f0ecd640 1 -- 172.21.15.7:0/3676726179 learned_addr learned my addr 172.21.15.7:0/3676726179 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1153862043 0 0) 0x7fe9ec136900 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9e9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9e9ffb640 10 monclient(hunting): my global_id is 124105 2024-05-08T15:53:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.256+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9d8003520 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3721770657 0 0) 0x7fe9d8003520 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe9ec136900 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe9e0001ad0 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): dump: 2024-05-08T15:53:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.277 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.278 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1842621715 0 0) 0x7fe9ec1d7ab0 con 0x7fe9ec137940 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): my global_id is 6287 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9d8003520 con 0x7fe9ec137940 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860858632 0 0) 0x7fe9ec135530 con 0x7fe9ec0a41b0 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient(hunting): my global_id is 6085 2024-05-08T15:53:27.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9ec1d7ab0 con 0x7fe9ec0a41b0 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4117830431 0 0) 0x7fe9ec136900 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.81:6791/0 conn(0x7fe9ec137940 legacy=0x7fe9ec1d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.81:6789/0 conn(0x7fe9ec0a41b0 legacy=0x7fe9ec1361f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 monclient: found mon.c 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec1d8c90 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9f3932640 5 monclient: authenticate success, global_id 124105 2024-05-08T15:53:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9f3932640 10 monclient: _renew_subs 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9f3932640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec1d7c80 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe9e0003940 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe9e0003eb0 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9e9ffb640 10 monclient: dump: 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9f3932640 10 monclient: _renew_subs 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9f3932640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fe9ec1d80f0 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.257+0000 7fe9eb7fe640 4 set_mon_vals no callback set 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fe9e0004050 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9f3932640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe9ec0a9130 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(293..293 src has 1..293) ==== 3898+0+0 (unknown 1971243629 0 0) 0x7fe9e0046f60 con 0x7fe9ec0a49a0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9f1ecf640 10 monclient: get_auth_request con 0x7fe9d8039c60 auth_method 0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 690351149 0 0) 0x7fe9ec1d7ab0 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 10 monclient(hunting): my global_id is 95535 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9ec136900 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 893511839 0 0) 0x7fe9ec136900 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe9ec1d7ab0 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe9dc001af0 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 10 monclient: dump: 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1532234148 0 0) 0x7fe9ec1d7ab0 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.258+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9ec0a0f60 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.259+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe9dc001af0 con 0x7fe9ec1d83e0 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.259+0000 7fe9e9ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.259+0000 7fe9e9ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.259+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.7:6789/0 conn(0x7fe9ec1d83e0 legacy=0x7fe9ec09ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.259+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.7:6789/0 conn(0x7fe9ec1d83e0 legacy=0x7fe9ec09ba50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.265+0000 7fe9f3932640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.265+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe9ec141cd0 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3387301274 0 0) 0x7fe9ec0a39f0 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient(hunting): my global_id is 95544 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9ec1d7ab0 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3389482072 0 0) 0x7fe9ec1d7ab0 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe9ec0a39f0 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe9dc005090 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient: dump: 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.287 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2705397362 0 0) 0x7fe9ec0a39f0 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe9ec137780 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1359795338) 0x7fe9dc007440 con 0x7fe9ec1d87c0 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.7:6789/0 conn(0x7fe9ec1d87c0 legacy=0x7fe9ec09c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.266+0000 7fe9e9ffb640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.7:6789/0 conn(0x7fe9ec1d87c0 legacy=0x7fe9ec09c160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:27.288 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 257817, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:27.289 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:27.290 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.290 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:53:27.290 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.290 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:27.290 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:27.290 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.291 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.292 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:27.293 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.294 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.295 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.296 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.297 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.298 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.299 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:27.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.301 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.301 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.301 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.302 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.306 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:27.308 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.309 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:27.310 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.81:6812/847627118 conn(0x7fe9d8039c60 legacy=0x7fe9d803c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 10 monclient: shutdown 2024-05-08T15:53:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 >> v1:172.21.15.7:6791/0 conn(0x7fe9ec0a49a0 legacy=0x7fe9ec1d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f1ecf640 1 -- 172.21.15.7:0/3676726179 reap_dead start 2024-05-08T15:53:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f1ecf640 1 -- 172.21.15.7:0/3676726179 reap_dead start 2024-05-08T15:53:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 shutdown_connections 2024-05-08T15:53:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 >> 172.21.15.7:0/3676726179 conn(0x7fe9ec0a9fa0 msgr2=0x7fe9ec1308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 shutdown_connections 2024-05-08T15:53:27.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.267+0000 7fe9f3932640 1 -- 172.21.15.7:0/3676726179 wait complete. 2024-05-08T15:53:27.314 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:27.314 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:53:27.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:53:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.422+0000 7fbe6cc3e640 10 monclient: get_monmap_and_config 2024-05-08T15:53:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.422+0000 7fbe6cc3e640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.422+0000 7fbe6cc3e640 10 monclient: monmap: 2024-05-08T15:53:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:27.423623+0000 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:27.423623+0000 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.419 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 1 Processor -- start 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 1 -- start start 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient: init 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient: _add_conns ranks=[7,4,8,6,1,0,5,3,2] 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient(hunting): picked mon.noname-i con 0x7fbe68137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient(hunting): picked mon.noname-c con 0x7fbe680a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient(hunting): picked mon.noname-e con 0x7fbe680a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe680a2c70 con 0x7fbe680a49a0 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe680a2f50 con 0x7fbe68137940 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe680a1670 con 0x7fbe680a41b0 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.423+0000 7fbe6cc3e640 10 monclient(hunting): authenticate will time out at 1215.538586s 2024-05-08T15:53:27.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe65d9b640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fbe680a49a0 0x7fbe6813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51808/0 (socket says 172.21.15.7:51808) 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe66d9d640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fbe680a41b0 0x7fbe68140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35526/0 (socket says 172.21.15.7:35526) 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe65d9b640 1 -- 172.21.15.7:0/3711165272 learned_addr learned my addr 172.21.15.7:0/3711165272 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683080612 0 0) 0x7fbe680a1670 con 0x7fbe680a41b0 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): my global_id is 95615 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe50003520 con 0x7fbe680a41b0 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1518038826 0 0) 0x7fbe680a2c70 con 0x7fbe680a49a0 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): my global_id is 124114 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe680a1670 con 0x7fbe680a49a0 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 852465877 0 0) 0x7fbe680a2f50 con 0x7fbe68137940 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient(hunting): my global_id is 114865 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe680a2c70 con 0x7fbe68137940 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2447274208 0 0) 0x7fbe50003520 con 0x7fbe680a41b0 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 >> v1:172.21.15.81:6792/0 conn(0x7fbe68137940 legacy=0x7fbe68137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 >> v1:172.21.15.7:6791/0 conn(0x7fbe680a49a0 legacy=0x7fbe6813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 monclient: found mon.noname-e 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:53:27.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe680a39f0 con 0x7fbe680a41b0 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6cc3e640 5 monclient: authenticate success, global_id 95615 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe60001a60 con 0x7fbe680a41b0 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6cc3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: dump: 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.432 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6cc3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbe60001cc0 con 0x7fbe680a41b0 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 1 -- 172.21.15.7:0/3711165272 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe60004ea0 con 0x7fbe680a41b0 2024-05-08T15:53:27.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6cc3e640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6cc3e640 4 set_mon_vals no callback set 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.424+0000 7fbe6559a640 10 monclient: dump: 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.434 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.425+0000 7fbe6cc3e640 10 monclient: shutdown 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.425+0000 7fbe6cc3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.425+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/3711165272 >> v1:172.21.15.7:6793/0 conn(0x7fbe680a41b0 legacy=0x7fbe68140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.425+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/3711165272 shutdown_connections 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.425+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/3711165272 >> 172.21.15.7:0/3711165272 conn(0x7fbe680a9fa0 msgr2=0x7fbe680aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.425+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/3711165272 shutdown_connections 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.425+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/3711165272 wait complete. 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient: monmap: 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.436 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 1 Processor -- start 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 1 -- start start 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient: init 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient: _add_conns ranks=[3,1,5,0,4,2,7,6,8] 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient(hunting): picked mon.noname-g con 0x7fbe680a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient(hunting): picked mon.noname-f con 0x7fbe680a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 10 monclient(hunting): picked mon.noname-h con 0x7fbe68137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.426+0000 7fbe6cc3e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe68133820 con 0x7fbe680a49a0 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe6cc3e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe68135ac0 con 0x7fbe680a41b0 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe6cc3e640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe681357e0 con 0x7fbe68137940 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe6cc3e640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe6cc3e640 10 monclient(hunting): authenticate will time out at 1215.541638s 2024-05-08T15:53:27.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe65d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbe680a49a0 0x7fbe681cc2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:55858/0 (socket says 172.21.15.7:55858) 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe6659c640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fbe680a41b0 0x7fbe68136ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:52786/0 (socket says 172.21.15.7:52786) 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe65d9b640 1 -- 172.21.15.7:0/1794608674 learned_addr learned my addr 172.21.15.7:0/1794608674 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe66d9d640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fbe68137940 0x7fbe681d4600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:42170/0 (socket says 172.21.15.7:42170) 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830437724 0 0) 0x7fbe68135ac0 con 0x7fbe680a41b0 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): my global_id is 76422 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe48003520 con 0x7fbe680a41b0 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479921077 0 0) 0x7fbe681357e0 con 0x7fbe68137940 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): my global_id is 6296 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe68135ac0 con 0x7fbe68137940 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3315382237 0 0) 0x7fbe68133820 con 0x7fbe680a49a0 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): my global_id is 6094 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe681357e0 con 0x7fbe680a49a0 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1829108330 0 0) 0x7fbe48003520 con 0x7fbe680a41b0 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe68133820 con 0x7fbe680a41b0 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe580019b0 con 0x7fbe680a41b0 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.439 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:27.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 114906170 0 0) 0x7fbe68135ac0 con 0x7fbe68137940 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe48003520 con 0x7fbe68137940 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 109691793 0 0) 0x7fbe681357e0 con 0x7fbe680a49a0 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe68135ac0 con 0x7fbe680a49a0 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe60004ea0 con 0x7fbe68137940 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.441 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe5c001510 con 0x7fbe680a49a0 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:27.442 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.443 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1920580246 0 0) 0x7fbe68133820 con 0x7fbe680a41b0 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.81:6791/0 conn(0x7fbe68137940 legacy=0x7fbe681d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.81:6789/0 conn(0x7fbe680a49a0 legacy=0x7fbe681cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.427+0000 7fbe56ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 1 monclient: found mon.g 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe68137700 con 0x7fbe680a41b0 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 5 monclient: authenticate success, global_id 76422 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 10 monclient: _renew_subs 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:53:27.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbe681cc9d0 con 0x7fbe680a41b0 2024-05-08T15:53:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 10 monclient: _renew_subs 2024-05-08T15:53:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:53:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fbe681d5d20 con 0x7fbe680a41b0 2024-05-08T15:53:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbe58001c70 con 0x7fbe680a41b0 2024-05-08T15:53:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe58003c00 con 0x7fbe680a41b0 2024-05-08T15:53:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe57fff640 4 set_mon_vals no callback set 2024-05-08T15:53:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe56ffd640 10 monclient: dump: 2024-05-08T15:53:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.446 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.428+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbe4c0065c0 con 0x7fbe4c002860 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fbe58003da0 con 0x7fbe680a41b0 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(293..293 src has 1..293) ==== 3898+0+0 (unknown 1971243629 0 0) 0x7fbe58046cc0 con 0x7fbe680a41b0 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904051558 0 0) 0x7fbe4c0065c0 con 0x7fbe4c002860 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 10 monclient(hunting): my global_id is 95600 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe68133820 con 0x7fbe4c002860 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe66d9d640 10 monclient: get_auth_request con 0x7fbe4803a500 auth_method 0 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 380020579 0 0) 0x7fbe68133820 con 0x7fbe4c002860 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe4c0065c0 con 0x7fbe4c002860 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe5c004b20 con 0x7fbe4c002860 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 10 monclient: dump: 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.449 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3886545200 0 0) 0x7fbe4c0065c0 con 0x7fbe4c002860 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.429+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe4c00a150 con 0x7fbe4c002860 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.430+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fbe5c005860 con 0x7fbe4c002860 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.430+0000 7fbe56ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.430+0000 7fbe56ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.430+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.7:6790/0 conn(0x7fbe4c002860 legacy=0x7fbe4c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.430+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.7:6790/0 conn(0x7fbe4c002860 legacy=0x7fbe4c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe6cc3e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbe4c009e40 con 0x7fbe4c009100 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1891373277 0 0) 0x7fbe4c009e40 con 0x7fbe4c009100 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe56ffd640 10 monclient(hunting): my global_id is 95609 2024-05-08T15:53:27.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.436+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe4c0065c0 con 0x7fbe4c009100 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2367489191 0 0) 0x7fbe4c0065c0 con 0x7fbe4c009100 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbe4c009e40 con 0x7fbe4c009100 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbe5c0054e0 con 0x7fbe4c009100 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 10 monclient: dump: 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.453 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 538070866 0 0) 0x7fbe4c009e40 con 0x7fbe4c009100 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbe4c00c3d0 con 0x7fbe4c009100 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1753197294) 0x7fbe5c007a70 con 0x7fbe4c009100 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:27.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.7:6790/0 conn(0x7fbe4c009100 legacy=0x7fbe4c009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe56ffd640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.7:6790/0 conn(0x7fbe4c009100 legacy=0x7fbe4c009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 257988, 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:27.455 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.456 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.457 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:27.458 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.459 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:27.460 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.461 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.462 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:27.463 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:27.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.466 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:27.468 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.470 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.471 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:27.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.473 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:27.474 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.81:6812/847627118 conn(0x7fbe4803a500 legacy=0x7fbe4803c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 10 monclient: shutdown 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 >> v1:172.21.15.81:6790/0 conn(0x7fbe680a41b0 legacy=0x7fbe68136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe66d9d640 1 -- 172.21.15.7:0/1794608674 reap_dead start 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe66d9d640 1 -- 172.21.15.7:0/1794608674 reap_dead start 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 shutdown_connections 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 >> 172.21.15.7:0/1794608674 conn(0x7fbe680a9fa0 msgr2=0x7fbe68130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 shutdown_connections 2024-05-08T15:53:27.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.437+0000 7fbe6cc3e640 1 -- 172.21.15.7:0/1794608674 wait complete. 2024-05-08T15:53:27.476 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:27.476 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:53:27.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.585+0000 7f743ec31640 10 monclient: get_monmap_and_config 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.585+0000 7f743ec31640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.585+0000 7f743ec31640 10 monclient: monmap: 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:27.586415+0000 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:27.586415+0000 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.581 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.582 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 1 Processor -- start 2024-05-08T15:53:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 1 -- start start 2024-05-08T15:53:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient: init 2024-05-08T15:53:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient: _add_conns ranks=[5,8,2,0,1,4,6,3,7] 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient(hunting): picked mon.noname-h con 0x7f7438137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient(hunting): picked mon.noname-e con 0x7f74380a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient(hunting): picked mon.noname-b con 0x7f74380a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74380a2c70 con 0x7f74380a41b0 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74380a2f50 con 0x7f7438137940 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74380a1670 con 0x7f74380a49a0 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f743ec31640 10 monclient(hunting): authenticate will time out at 1215.701493s 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f7437fff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f74380a49a0 0x7f743813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35536/0 (socket says 172.21.15.7:35536) 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.586+0000 7f7437fff640 1 -- 172.21.15.7:0/621694951 learned_addr learned my addr 172.21.15.7:0/621694951 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779017795 0 0) 0x7f74380a2c70 con 0x7f74380a41b0 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): my global_id is 95618 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7424003520 con 0x7f74380a41b0 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167996717 0 0) 0x7f74380a1670 con 0x7f74380a49a0 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient(hunting): my global_id is 95624 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f74380a2c70 con 0x7f74380a49a0 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3197590608 0 0) 0x7f7424003520 con 0x7f74380a41b0 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 >> v1:172.21.15.7:6793/0 conn(0x7f74380a49a0 legacy=0x7f743813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 >> v1:172.21.15.81:6791/0 conn(0x7f7438137940 legacy=0x7f7438137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 monclient: found mon.noname-b 2024-05-08T15:53:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74380a39f0 con 0x7f74380a41b0 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7430001b30 con 0x7f74380a41b0 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f743ec31640 5 monclient: authenticate success, global_id 95618 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f743ec31640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: dump: 2024-05-08T15:53:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f743ec31640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7430001d90 con 0x7f74380a41b0 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 1 -- 172.21.15.7:0/621694951 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7430003cd0 con 0x7f74380a41b0 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f743ec31640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f743ec31640 4 set_mon_vals no callback set 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.587+0000 7f74377fe640 10 monclient: dump: 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 10 monclient: shutdown 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 1 -- 172.21.15.7:0/621694951 >> v1:172.21.15.7:6790/0 conn(0x7f74380a41b0 legacy=0x7f7438140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 1 -- 172.21.15.7:0/621694951 shutdown_connections 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 1 -- 172.21.15.7:0/621694951 >> 172.21.15.7:0/621694951 conn(0x7f74380a9fa0 msgr2=0x7f74380aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 1 -- 172.21.15.7:0/621694951 shutdown_connections 2024-05-08T15:53:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 1 -- 172.21.15.7:0/621694951 wait complete. 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.588+0000 7f743ec31640 10 monclient: monmap: 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 1 Processor -- start 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 1 -- start start 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient: init 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient: _add_conns ranks=[6,0,3,4,2,7,8,5,1] 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient(hunting): picked mon.noname-d con 0x7f74380a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient(hunting): picked mon.noname-a con 0x7f74380a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient(hunting): picked mon.noname-g con 0x7f7438137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:27.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74380a1f30 con 0x7f74380a49a0 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74380a0f60 con 0x7f7438137940 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7438135440 con 0x7f74380a41b0 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743ec31640 10 monclient(hunting): authenticate will time out at 1215.704353s 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743c9cd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f74380a41b0 0x7f7438133a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35276/0 (socket says 172.21.15.7:35276) 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f7437fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f74380a49a0 0x7f74381d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39064/0 (socket says 172.21.15.7:39064) 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f743c9cd640 1 -- 172.21.15.7:0/52820954 learned_addr learned my addr 172.21.15.7:0/52820954 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.589+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385995345 0 0) 0x7f7428001940 con 0x7f74380a49a0 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): my global_id is 95553 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7418003520 con 0x7f74380a49a0 2024-05-08T15:53:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3181433506 0 0) 0x7f7438135440 con 0x7f74380a41b0 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): my global_id is 124116 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7428001940 con 0x7f74380a41b0 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2682179536 0 0) 0x7f7418003520 con 0x7f74380a49a0 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7438135440 con 0x7f74380a49a0 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7428001bd0 con 0x7f74380a49a0 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): dump: 2024-05-08T15:53:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3845338691 0 0) 0x7f7428001940 con 0x7f74380a41b0 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7418003520 con 0x7f74380a41b0 2024-05-08T15:53:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1657427429 0 0) 0x7f74380a0f60 con 0x7f7438137940 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): my global_id is 76431 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7428001940 con 0x7f7438137940 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7420001a60 con 0x7f74380a41b0 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient(hunting): dump: 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3039081615 0 0) 0x7f7438135440 con 0x7f74380a49a0 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.7:6792/0 conn(0x7f74380a41b0 legacy=0x7f7438133a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.81:6790/0 conn(0x7f7438137940 legacy=0x7f74381d4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 monclient: found mon.a 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7438137450 con 0x7f74380a49a0 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f743ec31640 5 monclient: authenticate success, global_id 95553 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f743ec31640 10 monclient: _renew_subs 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f743ec31640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7438135fa0 con 0x7f74380a49a0 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f743ec31640 10 monclient: _renew_subs 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f743ec31640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7438141800 con 0x7f74380a49a0 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7428004a90 con 0x7f74380a49a0 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7428004f80 con 0x7f74380a49a0 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f7436ffd640 4 set_mon_vals no callback set 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.590+0000 7f74357fa640 10 monclient: dump: 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.613 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7428010300 con 0x7f74380a49a0 2024-05-08T15:53:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f743ec31640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f741c0065c0 con 0x7f741c002860 2024-05-08T15:53:27.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(293..293 src has 1..293) ==== 3898+0+0 (unknown 1971243629 0 0) 0x7f7428046f40 con 0x7f74380a49a0 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f743c9cd640 10 monclient: get_auth_request con 0x7f741803a200 auth_method 0 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2349984683 0 0) 0x7f741c0065c0 con 0x7f741c002860 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 10 monclient(hunting): my global_id is 124123 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.591+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7438135440 con 0x7f741c002860 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2446069017 0 0) 0x7f7438135440 con 0x7f741c002860 2024-05-08T15:53:27.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f741c0065c0 con 0x7f741c002860 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7430001250 con 0x7f741c002860 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 10 monclient: dump: 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.615 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3952970650 0 0) 0x7f741c0065c0 con 0x7f741c002860 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f741c00a150 con 0x7f741c002860 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f7430001470 con 0x7f741c002860 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:27.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.7:6791/0 conn(0x7f741c002860 legacy=0x7f741c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.592+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.7:6791/0 conn(0x7f741c002860 legacy=0x7f741c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f743ec31640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f741c009e20 con 0x7f741c0090e0 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4084138733 0 0) 0x7f741c009e20 con 0x7f741c0090e0 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 10 monclient(hunting): my global_id is 124132 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f741c0065c0 con 0x7f741c0090e0 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 609161368 0 0) 0x7f741c0065c0 con 0x7f741c0090e0 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f741c009e20 con 0x7f741c0090e0 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7430003fe0 con 0x7f741c0090e0 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 10 monclient: dump: 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.620 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1033392149 0 0) 0x7f741c009e20 con 0x7f741c0090e0 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.599+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f741c00c390 con 0x7f741c0090e0 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7522 (unknown 0 0 1840865498) 0x7f74300062a0 con 0x7f741c0090e0 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f74357fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f74357fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.7:6791/0 conn(0x7f741c0090e0 legacy=0x7f741c009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f74357fa640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.7:6791/0 conn(0x7f741c0090e0 legacy=0x7f741c009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 15769, 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:27.622 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.623 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.624 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.625 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:27.626 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:27.627 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:27.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.629 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:27.630 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.631 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:27.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.637 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:27.637 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:27.637 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:27.638 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.639 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.639 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.639 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.640 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:27.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:27.640 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:27.641 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.642 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.642 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.642 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.642 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.643 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:27.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:27.643 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:27.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.644 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.645 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.645 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.645 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.645 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.646 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:27.646 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:27.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.646 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.646 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.646 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.647 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.647 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:27.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.647 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.647 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.648 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.648 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:27.648 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:27.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.81:6812/847627118 conn(0x7f741803a200 legacy=0x7f741803c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 10 monclient: shutdown 2024-05-08T15:53:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 >> v1:172.21.15.7:6789/0 conn(0x7f74380a49a0 legacy=0x7f74381d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743d1ce640 1 -- 172.21.15.7:0/52820954 reap_dead start 2024-05-08T15:53:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743d1ce640 1 -- 172.21.15.7:0/52820954 reap_dead start 2024-05-08T15:53:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 shutdown_connections 2024-05-08T15:53:27.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 >> 172.21.15.7:0/52820954 conn(0x7f74380a9fa0 msgr2=0x7f7438130760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 shutdown_connections 2024-05-08T15:53:27.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.600+0000 7f743ec31640 1 -- 172.21.15.7:0/52820954 wait complete. 2024-05-08T15:53:27.652 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:27.652 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:53:27.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:53:27.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.767+0000 7fb5dec9a640 10 monclient: get_monmap_and_config 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.767+0000 7fb5dec9a640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.767+0000 7fb5dec9a640 10 monclient: monmap: 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:27.768745+0000 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:27.768745+0000 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.764 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.765 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.766 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.768+0000 7fb5dec9a640 1 Processor -- start 2024-05-08T15:53:27.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.768+0000 7fb5dec9a640 1 -- start start 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.768+0000 7fb5dec9a640 10 monclient: init 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 10 monclient: _add_conns ranks=[3,1,5,7,6,2,8,0,4] 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 10 monclient(hunting): picked mon.noname-g con 0x7fb5d80a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 10 monclient(hunting): picked mon.noname-f con 0x7fb5d80a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 10 monclient(hunting): picked mon.noname-h con 0x7fb5d813e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d80a1660 con 0x7fb5d80a5780 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d80a28f0 con 0x7fb5d80a53a0 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d80a2bd0 con 0x7fb5d813e310 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dec9a640 10 monclient(hunting): authenticate will time out at 1215.883855s 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d7fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb5d80a5780 0x7fb5d813cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:55868/0 (socket says 172.21.15.7:55868) 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dca36640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fb5d80a53a0 0x7fb5d80a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:52800/0 (socket says 172.21.15.7:52800) 2024-05-08T15:53:27.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d7fff640 1 -- 172.21.15.7:0/2764583939 learned_addr learned my addr 172.21.15.7:0/2764583939 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5dd237640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7fb5d813e310 0x7fb5d8140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:42186/0 (socket says 172.21.15.7:42186) 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1246385988 0 0) 0x7fb5d80a1660 con 0x7fb5d80a5780 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): my global_id is 6103 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5c8003520 con 0x7fb5d80a5780 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2312362286 0 0) 0x7fb5d80a2bd0 con 0x7fb5d813e310 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): my global_id is 6314 2024-05-08T15:53:27.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d80a1660 con 0x7fb5d813e310 2024-05-08T15:53:27.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3557345079 0 0) 0x7fb5d80a28f0 con 0x7fb5d80a53a0 2024-05-08T15:53:27.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient(hunting): my global_id is 76440 2024-05-08T15:53:27.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d80a2bd0 con 0x7fb5d80a53a0 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3797440110 0 0) 0x7fb5c8003520 con 0x7fb5d80a5780 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 >> v1:172.21.15.81:6791/0 conn(0x7fb5d813e310 legacy=0x7fb5d8140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 >> v1:172.21.15.81:6790/0 conn(0x7fb5d80a53a0 legacy=0x7fb5d80a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 monclient: found mon.noname-f 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient: _send_mon_message to mon.noname-f at v1:172.21.15.81:6789/0 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d80a6730 con 0x7fb5d80a5780 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5c40017b0 con 0x7fb5d80a5780 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient: got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.769+0000 7fb5d77fe640 10 monclient: dump: 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.779 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 5 monclient: authenticate success, global_id 6103 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 <== mon.1 v1:172.21.15.81:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb5c4001ad0 con 0x7fb5d80a5780 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5d77fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5d77fe640 1 -- 172.21.15.7:0/2764583939 <== mon.1 v1:172.21.15.81:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5c4004be0 con 0x7fb5d80a5780 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 4 set_mon_vals no callback set 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5d77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5d77fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5d77fe640 10 monclient: dump: 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.781 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 10 monclient: shutdown 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 1 -- 172.21.15.7:0/2764583939 >> v1:172.21.15.81:6789/0 conn(0x7fb5d80a5780 legacy=0x7fb5d813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 1 -- 172.21.15.7:0/2764583939 shutdown_connections 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 1 -- 172.21.15.7:0/2764583939 >> 172.21.15.7:0/2764583939 conn(0x7fb5d80a9fa0 msgr2=0x7fb5d80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 1 -- 172.21.15.7:0/2764583939 shutdown_connections 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.770+0000 7fb5dec9a640 1 -- 172.21.15.7:0/2764583939 wait complete. 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.771+0000 7fb5dec9a640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.771+0000 7fb5dec9a640 10 monclient: monmap: 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.783 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.771+0000 7fb5dec9a640 1 Processor -- start 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.771+0000 7fb5dec9a640 1 -- start start 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.771+0000 7fb5dec9a640 10 monclient: init 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 10 monclient: _add_conns ranks=[5,4,6,8,2,0,7,3,1] 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 10 monclient(hunting): picked mon.noname-h con 0x7fb5d80a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 10 monclient(hunting): picked mon.noname-c con 0x7fb5d80a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 10 monclient(hunting): picked mon.noname-d con 0x7fb5d813e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d80a1e00 con 0x7fb5d80a5780 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d813ca70 con 0x7fb5d80a53a0 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d81344e0 con 0x7fb5d813e310 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dec9a640 10 monclient(hunting): authenticate will time out at 1215.887019s 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dd237640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fb5d813e310 0x7fb5d81d4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35286/0 (socket says 172.21.15.7:35286) 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5dd237640 1 -- 172.21.15.7:0/3643191238 learned_addr learned my addr 172.21.15.7:0/3643191238 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919512376 0 0) 0x7fb5d80a1e00 con 0x7fb5d80a5780 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): my global_id is 124141 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5bc003520 con 0x7fb5d80a5780 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706888394 0 0) 0x7fb5d81344e0 con 0x7fb5d813e310 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): my global_id is 124125 2024-05-08T15:53:27.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d80a1e00 con 0x7fb5d813e310 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1768027739 0 0) 0x7fb5d813ca70 con 0x7fb5d80a53a0 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 10 monclient(hunting): my global_id is 6323 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.772+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d81344e0 con 0x7fb5d80a53a0 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1651154348 0 0) 0x7fb5bc003520 con 0x7fb5d80a5780 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb5d813ca70 con 0x7fb5d80a5780 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5c4004b20 con 0x7fb5d80a5780 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): dump: 2024-05-08T15:53:27.786 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.787 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1998673407 0 0) 0x7fb5d80a1e00 con 0x7fb5d813e310 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb5bc003520 con 0x7fb5d813e310 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4194203116 0 0) 0x7fb5d81344e0 con 0x7fb5d80a53a0 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb5d80a1e00 con 0x7fb5d80a53a0 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5d0001c20 con 0x7fb5d813e310 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): dump: 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.788 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.789 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5cc001910 con 0x7fb5d80a53a0 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient(hunting): dump: 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.790 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 922378037 0 0) 0x7fb5d813ca70 con 0x7fb5d80a5780 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.7:6792/0 conn(0x7fb5d813e310 legacy=0x7fb5d81d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.81:6791/0 conn(0x7fb5d80a53a0 legacy=0x7fb5d813bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 monclient: found mon.c 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d81347c0 con 0x7fb5d80a5780 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5dec9a640 5 monclient: authenticate success, global_id 124141 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5dec9a640 10 monclient: _renew_subs 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5dec9a640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d8138750 con 0x7fb5d80a5780 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5dec9a640 10 monclient: _renew_subs 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5dec9a640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fb5d81337f0 con 0x7fb5d80a5780 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fb5c4001850 con 0x7fb5d80a5780 2024-05-08T15:53:27.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5c40051a0 con 0x7fb5d80a5780 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d67fc640 4 set_mon_vals no callback set 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.773+0000 7fb5d57fa640 10 monclient: dump: 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.793 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.794 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5dec9a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fb5c4010520 con 0x7fb5d80a5780 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb5b80065c0 con 0x7fb5b8002860 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(293..293 src has 1..293) ==== 3898+0+0 (unknown 1971243629 0 0) 0x7fb5c4046ea0 con 0x7fb5d80a5780 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 903411533 0 0) 0x7fb5b80065c0 con 0x7fb5b8002860 2024-05-08T15:53:27.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 10 monclient(hunting): my global_id is 124134 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d813ca70 con 0x7fb5b8002860 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.774+0000 7fb5dd237640 10 monclient: get_auth_request con 0x7fb5bc03a560 auth_method 0 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 951013315 0 0) 0x7fb5d813ca70 con 0x7fb5b8002860 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb5b80065c0 con 0x7fb5b8002860 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5cc004b20 con 0x7fb5b8002860 2024-05-08T15:53:27.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 10 monclient: dump: 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.797 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3727867452 0 0) 0x7fb5b80065c0 con 0x7fb5b8002860 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb5b800a150 con 0x7fb5b8002860 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fb5cc004eb0 con 0x7fb5b8002860 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.7:6792/0 conn(0x7fb5b8002860 legacy=0x7fb5b8004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.775+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.7:6792/0 conn(0x7fb5b8002860 legacy=0x7fb5b8004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.784+0000 7fb5dec9a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fb5b8009e40 con 0x7fb5b8009100 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3501822822 0 0) 0x7fb5b8009e40 con 0x7fb5b8009100 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 10 monclient(hunting): my global_id is 124143 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5b80065c0 con 0x7fb5b8009100 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2404374087 0 0) 0x7fb5b80065c0 con 0x7fb5b8009100 2024-05-08T15:53:27.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fb5b8009e40 con 0x7fb5b8009100 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fb5cc0056f0 con 0x7fb5b8009100 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 10 monclient: dump: 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.801 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 697201883 0 0) 0x7fb5b8009e40 con 0x7fb5b8009100 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb5b800c3d0 con 0x7fb5b8009100 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.785+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7522 (unknown 0 0 1851095786) 0x7fb5cc006aa0 con 0x7fb5b8009100 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5d57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5d57fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.7:6792/0 conn(0x7fb5b8009100 legacy=0x7fb5b8009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5d57fa640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.7:6792/0 conn(0x7fb5b8009100 legacy=0x7fb5b8009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 15955, 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:27.803 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:27.805 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:27.806 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:27.807 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:27.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.810 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:27.811 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.813 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:27.814 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.815 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.816 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:27.817 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:27.818 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.819 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.820 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.821 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:27.822 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.81:6812/847627118 conn(0x7fb5bc03a560 legacy=0x7fb5bc03ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 10 monclient: shutdown 2024-05-08T15:53:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 >> v1:172.21.15.7:6791/0 conn(0x7fb5d80a5780 legacy=0x7fb5d813c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dd237640 1 -- 172.21.15.7:0/3643191238 reap_dead start 2024-05-08T15:53:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dd237640 1 -- 172.21.15.7:0/3643191238 reap_dead start 2024-05-08T15:53:27.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 shutdown_connections 2024-05-08T15:53:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 >> 172.21.15.7:0/3643191238 conn(0x7fb5d80a9fa0 msgr2=0x7fb5d80a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 shutdown_connections 2024-05-08T15:53:27.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.786+0000 7fb5dec9a640 1 -- 172.21.15.7:0/3643191238 wait complete. 2024-05-08T15:53:27.825 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:27.825 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:53:27.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:53:27.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.943+0000 7f53ea3b6640 10 monclient: get_monmap_and_config 2024-05-08T15:53:27.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.943+0000 7f53ea3b6640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.943+0000 7f53ea3b6640 10 monclient: monmap: 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:27.944708+0000 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:27.944708+0000 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.940 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.941 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.944+0000 7f53ea3b6640 1 Processor -- start 2024-05-08T15:53:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.944+0000 7f53ea3b6640 1 -- start start 2024-05-08T15:53:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.944+0000 7f53ea3b6640 10 monclient: init 2024-05-08T15:53:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient: _add_conns ranks=[8,7,2,6,0,1,5,4,3] 2024-05-08T15:53:27.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient(hunting): picked mon.noname-e con 0x7f53e4137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient(hunting): picked mon.noname-i con 0x7f53e40a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient(hunting): picked mon.noname-b con 0x7f53e40a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40a2c70 con 0x7f53e40a41b0 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40a2f50 con 0x7f53e40a49a0 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40a1670 con 0x7f53e4137940 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient(hunting): authenticate will time out at 1216.059785s 2024-05-08T15:53:27.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e3fff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f53e4137940 0x7f53e4137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35540/0 (socket says 172.21.15.7:35540) 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e8953640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f53e40a41b0 0x7f53e4140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51928/0 (socket says 172.21.15.7:51928) 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e3fff640 1 -- 172.21.15.7:0/1294031256 learned_addr learned my addr 172.21.15.7:0/1294031256 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1624629257 0 0) 0x7f53e40a1670 con 0x7f53e4137940 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): my global_id is 95633 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53d0003520 con 0x7f53e4137940 2024-05-08T15:53:27.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223483318 0 0) 0x7f53e40a2c70 con 0x7f53e40a41b0 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient(hunting): my global_id is 95627 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e40a1670 con 0x7f53e40a41b0 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 465020865 0 0) 0x7f53e40a1670 con 0x7f53e40a41b0 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 >> v1:172.21.15.7:6793/0 conn(0x7f53e4137940 legacy=0x7f53e4137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 >> v1:172.21.15.81:6792/0 conn(0x7f53e40a49a0 legacy=0x7f53e413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 monclient: found mon.noname-b 2024-05-08T15:53:27.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e40a39f0 con 0x7f53e40a41b0 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53dc001a60 con 0x7f53e40a41b0 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 5 monclient: authenticate success, global_id 95627 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: dump: 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.957 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f53dc001d40 con 0x7f53e40a41b0 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53e37fe640 1 -- 172.21.15.7:0/1294031256 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53dc004ea0 con 0x7f53e40a41b0 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.945+0000 7f53ea3b6640 4 set_mon_vals no callback set 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53e37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53e37fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:27.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53e37fe640 10 monclient: dump: 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.959 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53ea3b6640 10 monclient: shutdown 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53ea3b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1294031256 >> v1:172.21.15.7:6790/0 conn(0x7f53e40a41b0 legacy=0x7f53e4140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1294031256 shutdown_connections 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1294031256 >> 172.21.15.7:0/1294031256 conn(0x7f53e40a9fa0 msgr2=0x7f53e40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1294031256 shutdown_connections 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.946+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1294031256 wait complete. 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient: build_initial_monmap 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient: monmap: 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:27.960 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 1 Processor -- start 2024-05-08T15:53:27.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 1 -- start start 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient: init 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient: _add_conns ranks=[2,6,5,3,7,1,0,8,4] 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient(hunting): picked mon.noname-b con 0x7f53e40a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient(hunting): picked mon.noname-d con 0x7f53e40a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient(hunting): picked mon.noname-h con 0x7f53e4137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e4135d50 con 0x7f53e40a41b0 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40a1f30 con 0x7f53e4137940 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53e40a0f60 con 0x7f53e40a49a0 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53ea3b6640 10 monclient(hunting): authenticate will time out at 1216.062573s 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.947+0000 7f53e3fff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f53e40a41b0 0x7f53e4131e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51940/0 (socket says 172.21.15.7:51940) 2024-05-08T15:53:27.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e3fff640 1 -- 172.21.15.7:0/1259910599 learned_addr learned my addr 172.21.15.7:0/1259910599 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 206656465 0 0) 0x7f53e4135d50 con 0x7f53e40a41b0 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): my global_id is 95636 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c4003520 con 0x7f53e40a41b0 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243435633 0 0) 0x7f53e40a0f60 con 0x7f53e40a49a0 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): my global_id is 124152 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e4135d50 con 0x7f53e40a49a0 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 143914852 0 0) 0x7f53e40a1f30 con 0x7f53e4137940 2024-05-08T15:53:27.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): my global_id is 6332 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e40a0f60 con 0x7f53e4137940 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1496277495 0 0) 0x7f53c4003520 con 0x7f53e40a41b0 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53e40a1f30 con 0x7f53e40a41b0 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53d40019c0 con 0x7f53e40a41b0 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): dump: 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.964 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1799395003 0 0) 0x7f53e4135d50 con 0x7f53e40a49a0 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53c4003520 con 0x7f53e40a49a0 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53cc0019d0 con 0x7f53e40a49a0 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient(hunting): dump: 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.966 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2050338844 0 0) 0x7f53e40a1f30 con 0x7f53e40a41b0 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.7:6792/0 conn(0x7f53e40a49a0 legacy=0x7f53e4132520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.81:6791/0 conn(0x7f53e4137940 legacy=0x7f53e41d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 monclient: found mon.b 2024-05-08T15:53:27.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e4132c30 con 0x7f53e40a41b0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53ea3b6640 5 monclient: authenticate success, global_id 95636 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53ea3b6640 10 monclient: _renew_subs 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53ea3b6640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f53e4135a70 con 0x7f53e40a41b0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53ea3b6640 10 monclient: _renew_subs 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53ea3b6640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f53e4137670 con 0x7f53e40a41b0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f53d4004a90 con 0x7f53e40a41b0 2024-05-08T15:53:27.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53d4004f80 con 0x7f53e40a41b0 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e2ffd640 4 set_mon_vals no callback set 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.948+0000 7f53e17fa640 10 monclient: dump: 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.969 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.949+0000 7f53ea3b6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.949+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f53c80065c0 con 0x7f53c8002860 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.949+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f53d4010300 con 0x7f53e40a41b0 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.949+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(294..294 src has 1..294) ==== 3898+0+0 (unknown 1267952350 0 0) 0x7f53d4046d60 con 0x7f53e40a41b0 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 996930121 0 0) 0x7f53c80065c0 con 0x7f53c8002860 2024-05-08T15:53:27.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient(hunting): my global_id is 95642 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53e40a1f30 con 0x7f53c8002860 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e8953640 10 monclient: get_auth_request con 0x7f53c4039ef0 auth_method 0 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2042037056 0 0) 0x7f53e40a1f30 con 0x7f53c8002860 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53c80065c0 con 0x7f53c8002860 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53cc003950 con 0x7f53c8002860 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient: dump: 2024-05-08T15:53:27.971 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.972 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1110941137 0 0) 0x7f53c80065c0 con 0x7f53c8002860 2024-05-08T15:53:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53c800a150 con 0x7f53c8002860 2024-05-08T15:53:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f53cc004640 con 0x7f53c8002860 2024-05-08T15:53:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.7:6793/0 conn(0x7f53c8002860 legacy=0x7f53c8004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.950+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.7:6793/0 conn(0x7f53c8002860 legacy=0x7f53c8004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53ea3b6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f53c8009e40 con 0x7f53c8009100 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2507170574 0 0) 0x7f53c8009e40 con 0x7f53c8009100 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 10 monclient(hunting): my global_id is 95651 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c80065c0 con 0x7f53c8009100 2024-05-08T15:53:27.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2329105879 0 0) 0x7f53c80065c0 con 0x7f53c8009100 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53c8009e40 con 0x7f53c8009100 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53cc004280 con 0x7f53c8009100 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 10 monclient: dump: 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:27.976 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 963436401 0 0) 0x7f53c8009e40 con 0x7f53c8009100 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.957+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f53c800c3d0 con 0x7f53c8009100 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 2036815685) 0x7f53cc0047f0 con 0x7f53c8009100 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53e17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:27.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53e17fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.7:6793/0 conn(0x7f53c8009100 legacy=0x7f53c8009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53e17fa640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.7:6793/0 conn(0x7f53c8009100 legacy=0x7f53c8009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:27.978 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 258509, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:27.979 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.980 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:27.982 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:27.983 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.984 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.985 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.987 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.988 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.993 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.996 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:27.997 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.998 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:27.999 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.002 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.002 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.002 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.002 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:28.002 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.81:6812/847627118 conn(0x7f53c4039ef0 legacy=0x7f53c403c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 10 monclient: shutdown 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 >> v1:172.21.15.7:6790/0 conn(0x7f53e40a41b0 legacy=0x7f53e4131e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53e8953640 1 -- 172.21.15.7:0/1259910599 reap_dead start 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53e8953640 1 -- 172.21.15.7:0/1259910599 reap_dead start 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 shutdown_connections 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 >> 172.21.15.7:0/1259910599 conn(0x7f53e40a9fa0 msgr2=0x7f53e4130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 shutdown_connections 2024-05-08T15:53:28.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:27.958+0000 7f53ea3b6640 1 -- 172.21.15.7:0/1259910599 wait complete. 2024-05-08T15:53:28.005 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:28.005 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:53:28.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:53:28.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.119+0000 7f944bef2640 10 monclient: get_monmap_and_config 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.119+0000 7f944bef2640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.119+0000 7f944bef2640 10 monclient: monmap: 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:28.120058+0000 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:28.120058+0000 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.115 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.116 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.116 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.116 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.116 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.116 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.116 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.116 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.119+0000 7f944bef2640 1 Processor -- start 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 1 -- start start 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient: init 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient: _add_conns ranks=[6,0,2,3,5,7,1,4,8] 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient(hunting): picked mon.noname-d con 0x7f94440a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient(hunting): picked mon.noname-a con 0x7f94440a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient(hunting): picked mon.noname-b con 0x7f944413e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94440a1660 con 0x7f94440a5780 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94440a28f0 con 0x7f944413e310 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94440a2bd0 con 0x7f94440a53a0 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944bef2640 10 monclient(hunting): authenticate will time out at 1216.235303s 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f9449c8e640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f94440a53a0 0x7f94440a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35328/0 (socket says 172.21.15.7:35328) 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f9449c8e640 1 -- 172.21.15.7:0/3674452499 learned_addr learned my addr 172.21.15.7:0/3674452499 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.120+0000 7f944a48f640 1 --1- 172.21.15.7:0/3674452499 >> v1:172.21.15.7:6790/0 conn(0x7f944413e310 0x7f9444140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51948/0 (socket says 172.21.15.7:51948) 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127343490 0 0) 0x7f94440a2bd0 con 0x7f94440a53a0 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): my global_id is 124161 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9430003520 con 0x7f94440a53a0 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228254958 0 0) 0x7f94440a1660 con 0x7f94440a5780 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): my global_id is 95562 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94440a2bd0 con 0x7f94440a5780 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1141797303 0 0) 0x7f94440a28f0 con 0x7f944413e310 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient(hunting): my global_id is 95645 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94440a1660 con 0x7f944413e310 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3560346839 0 0) 0x7f9430003520 con 0x7f94440a53a0 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 >> v1:172.21.15.7:6790/0 conn(0x7f944413e310 legacy=0x7f9444140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 >> v1:172.21.15.7:6789/0 conn(0x7f94440a5780 legacy=0x7f944413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 monclient: found mon.noname-d 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94440a6730 con 0x7f94440a53a0 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f94400017b0 con 0x7f94440a53a0 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: dump: 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.137 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f944bef2640 5 monclient: authenticate success, global_id 124161 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f944bef2640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9440001a10 con 0x7f94440a53a0 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 1 -- 172.21.15.7:0/3674452499 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9440004be0 con 0x7f94440a53a0 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f944bef2640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f944bef2640 4 set_mon_vals no callback set 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.121+0000 7f9448c8c640 10 monclient: dump: 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.139 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.122+0000 7f944bef2640 10 monclient: shutdown 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.122+0000 7f944bef2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.122+0000 7f944bef2640 1 -- 172.21.15.7:0/3674452499 >> v1:172.21.15.7:6792/0 conn(0x7f94440a53a0 legacy=0x7f94440a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.122+0000 7f944bef2640 1 -- 172.21.15.7:0/3674452499 shutdown_connections 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.122+0000 7f944bef2640 1 -- 172.21.15.7:0/3674452499 >> 172.21.15.7:0/3674452499 conn(0x7f94440a9fa0 msgr2=0x7f94440aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.122+0000 7f944bef2640 1 -- 172.21.15.7:0/3674452499 shutdown_connections 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.122+0000 7f944bef2640 1 -- 172.21.15.7:0/3674452499 wait complete. 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient: monmap: 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.141 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 1 Processor -- start 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 1 -- start start 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient: init 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient: _add_conns ranks=[6,3,1,2,7,4,5,8,0] 2024-05-08T15:53:28.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient(hunting): picked mon.noname-d con 0x7f94440a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient(hunting): picked mon.noname-g con 0x7f94440a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 10 monclient(hunting): picked mon.noname-f con 0x7f944413e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f944413af30 con 0x7f944413e310 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.123+0000 7f944bef2640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9444131510 con 0x7f94440a5780 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94440a1e00 con 0x7f94440a53a0 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f9449c8e640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f94440a53a0 0x7f94441306f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35332/0 (socket says 172.21.15.7:35332) 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 10 monclient(hunting): authenticate will time out at 1216.238651s 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f9449c8e640 1 -- 172.21.15.7:0/2379417800 learned_addr learned my addr 172.21.15.7:0/2379417800 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1139402310 0 0) 0x7f94440a1e00 con 0x7f94440a53a0 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): my global_id is 124170 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9428003520 con 0x7f94440a53a0 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2445844750 0 0) 0x7f9444131510 con 0x7f94440a5780 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): my global_id is 76449 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94440a1e00 con 0x7f94440a5780 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 526685020 0 0) 0x7f944413af30 con 0x7f944413e310 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): my global_id is 6112 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9444131510 con 0x7f944413e310 2024-05-08T15:53:28.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 197406371 0 0) 0x7f9428003520 con 0x7f94440a53a0 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f944413af30 con 0x7f94440a53a0 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9440004b20 con 0x7f94440a53a0 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): dump: 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.145 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3299564565 0 0) 0x7f944413af30 con 0x7f94440a53a0 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.81:6790/0 conn(0x7f94440a5780 legacy=0x7f9444130e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.81:6789/0 conn(0x7f944413e310 legacy=0x7f94441d4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 monclient: found mon.d 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944413a7e0 con 0x7f94440a53a0 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 5 monclient: authenticate success, global_id 124170 2024-05-08T15:53:28.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 10 monclient: _renew_subs 2024-05-08T15:53:28.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:28.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f944413c7f0 con 0x7f94440a53a0 2024-05-08T15:53:28.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 10 monclient: _renew_subs 2024-05-08T15:53:28.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:28.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f944413e050 con 0x7f94440a53a0 2024-05-08T15:53:28.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9440004ef0 con 0x7f94440a53a0 2024-05-08T15:53:28.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9440001a10 con 0x7f94440a53a0 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943bfff640 4 set_mon_vals no callback set 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.124+0000 7f943a7fc640 10 monclient: dump: 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.149 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.150 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.125+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f94400105c0 con 0x7f94440a53a0 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.125+0000 7f944bef2640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.125+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(294..294 src has 1..294) ==== 3898+0+0 (unknown 1267952350 0 0) 0x7f9440046ef0 con 0x7f94440a53a0 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.125+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f942c0065c0 con 0x7f942c002860 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.125+0000 7f944948d640 10 monclient: get_auth_request con 0x7f9428039d30 auth_method 0 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 408780887 0 0) 0x7f942c0065c0 con 0x7f942c002860 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 10 monclient(hunting): my global_id is 6121 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f944413af30 con 0x7f942c002860 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 83002837 0 0) 0x7f944413af30 con 0x7f942c002860 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f942c0065c0 con 0x7f942c002860 2024-05-08T15:53:28.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f943c001c10 con 0x7f942c002860 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 10 monclient: dump: 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.152 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 309532037 0 0) 0x7f942c0065c0 con 0x7f942c002860 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f942c00a150 con 0x7f942c002860 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.126+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f943c001f90 con 0x7f942c002860 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.127+0000 7f943a7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.127+0000 7f943a7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.127+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.81:6789/0 conn(0x7f942c002860 legacy=0x7f942c004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.127+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.81:6789/0 conn(0x7f942c002860 legacy=0x7f942c004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.133+0000 7f944bef2640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.133+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f942c009e40 con 0x7f942c009100 2024-05-08T15:53:28.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3185435648 0 0) 0x7f942c009e40 con 0x7f942c009100 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 10 monclient(hunting): my global_id is 6130 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f942c0065c0 con 0x7f942c009100 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2311372487 0 0) 0x7f942c0065c0 con 0x7f942c009100 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f942c009e40 con 0x7f942c009100 2024-05-08T15:53:28.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f943c005090 con 0x7f942c009100 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 10 monclient: dump: 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.155 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.156 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.156 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.156 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.156 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.156 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.156 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.156 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 376828646 0 0) 0x7f942c009e40 con 0x7f942c009100 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.134+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f942c00c3d0 con 0x7f942c009100 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7695 (unknown 0 0 3900040621) 0x7f943c007440 con 0x7f942c009100 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f943a7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f943a7fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.81:6789/0 conn(0x7f942c009100 legacy=0x7f942c009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f943a7fc640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.81:6789/0 conn(0x7f942c009100 legacy=0x7f942c009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.158 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:28.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:28.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:28.158 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:53:28.158 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:28.158 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 400450, 2024-05-08T15:53:28.158 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:28.159 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.160 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:28.161 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:28.162 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.163 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.164 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.165 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:28.166 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.167 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:28.168 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.169 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.170 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.171 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.173 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:28.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.179 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.181 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:28.181 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:28.182 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38869]: starting. 2024-05-08T15:53:28.182 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38869]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38868 2024-05-08T15:53:28.182 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38869]: created object 0... 2024-05-08T15:53:28.182 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38869]: created object 25... 2024-05-08T15:53:28.182 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38869]: created object 49... 2024-05-08T15:53:28.182 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38869]: finishing. 2024-05-08T15:53:28.182 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38869]: shutting down. 2024-05-08T15:53:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.81:6812/847627118 conn(0x7f9428039d30 legacy=0x7f942803c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 10 monclient: shutdown 2024-05-08T15:53:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 >> v1:172.21.15.7:6792/0 conn(0x7f94440a53a0 legacy=0x7f94441306f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944a48f640 1 -- 172.21.15.7:0/2379417800 reap_dead start 2024-05-08T15:53:28.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 shutdown_connections 2024-05-08T15:53:28.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 >> 172.21.15.7:0/2379417800 conn(0x7f94440a9fa0 msgr2=0x7f9444136fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 shutdown_connections 2024-05-08T15:53:28.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.135+0000 7f944bef2640 1 -- 172.21.15.7:0/2379417800 wait complete. 2024-05-08T15:53:28.184 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:28.184 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:53:28.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:53:28.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.292+0000 7fd702d75640 10 monclient: get_monmap_and_config 2024-05-08T15:53:28.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.292+0000 7fd702d75640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.292+0000 7fd702d75640 10 monclient: monmap: 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:28.293888+0000 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:28.293888+0000 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.289 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.290 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.290 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.290 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.293+0000 7fd702d75640 1 Processor -- start 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.293+0000 7fd702d75640 1 -- start start 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.293+0000 7fd702d75640 10 monclient: init 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 10 monclient: _add_conns ranks=[7,2,4,6,5,1,0,3,8] 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 10 monclient(hunting): picked mon.noname-i con 0x7fd6fc137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 10 monclient(hunting): picked mon.noname-b con 0x7fd6fc0a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 10 monclient(hunting): picked mon.noname-c con 0x7fd6fc0a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc0a2c70 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc0a2f50 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc0a1670 con 0x7fd6fc137940 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 10 monclient(hunting): authenticate will time out at 1216.408832s 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fbfff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd6fc0a49a0 0x7fd6fc13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51956/0 (socket says 172.21.15.7:51956) 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd701312640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fd6fc0a41b0 0x7fd6fc140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51832/0 (socket says 172.21.15.7:51832) 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fbfff640 1 -- 172.21.15.7:0/88618318 learned_addr learned my addr 172.21.15.7:0/88618318 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671165486 0 0) 0x7fd6fc0a2c70 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): my global_id is 95654 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6e8003520 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3443160170 0 0) 0x7fd6fc0a2f50 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): my global_id is 124150 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6fc0a2c70 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410603598 0 0) 0x7fd6fc0a1670 con 0x7fd6fc137940 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient(hunting): my global_id is 114883 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6fc0a2f50 con 0x7fd6fc137940 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3322473283 0 0) 0x7fd6fc0a2c70 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 >> v1:172.21.15.81:6792/0 conn(0x7fd6fc137940 legacy=0x7fd6fc137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 >> v1:172.21.15.7:6790/0 conn(0x7fd6fc0a49a0 legacy=0x7fd6fc13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 monclient: found mon.noname-c 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc0a39f0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6ec001b10 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd702d75640 5 monclient: authenticate success, global_id 124150 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.294+0000 7fd6fb7fe640 10 monclient: dump: 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.301 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd6ec001df0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd6fb7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd6fb7fe640 1 -- 172.21.15.7:0/88618318 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6ec004ea0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 4 set_mon_vals no callback set 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd6fb7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd6fb7fe640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd6fb7fe640 10 monclient: dump: 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 10 monclient: shutdown 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 1 -- 172.21.15.7:0/88618318 >> v1:172.21.15.7:6791/0 conn(0x7fd6fc0a41b0 legacy=0x7fd6fc140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 1 -- 172.21.15.7:0/88618318 shutdown_connections 2024-05-08T15:53:28.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 1 -- 172.21.15.7:0/88618318 >> 172.21.15.7:0/88618318 conn(0x7fd6fc0a9fa0 msgr2=0x7fd6fc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 1 -- 172.21.15.7:0/88618318 shutdown_connections 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.295+0000 7fd702d75640 1 -- 172.21.15.7:0/88618318 wait complete. 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.296+0000 7fd702d75640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.296+0000 7fd702d75640 10 monclient: monmap: 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.305 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.296+0000 7fd702d75640 1 Processor -- start 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.296+0000 7fd702d75640 1 -- start start 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.296+0000 7fd702d75640 10 monclient: init 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 10 monclient: _add_conns ranks=[6,2,3,7,5,4,8,0,1] 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 10 monclient(hunting): picked mon.noname-d con 0x7fd6fc0a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 10 monclient(hunting): picked mon.noname-b con 0x7fd6fc0a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 10 monclient(hunting): picked mon.noname-g con 0x7fd6fc137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc1340f0 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc1359b0 con 0x7fd6fc137940 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6fc0a0f60 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd702d75640 10 monclient(hunting): authenticate will time out at 1216.411818s 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6fbfff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd6fc0a49a0 0x7fd6fc132500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51972/0 (socket says 172.21.15.7:51972) 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6fbfff640 1 -- 172.21.15.7:0/4228753062 learned_addr learned my addr 172.21.15.7:0/4228753062 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd700b11640 1 --1- 172.21.15.7:0/4228753062 >> v1:172.21.15.7:6792/0 conn(0x7fd6fc0a41b0 0x7fd6fc131df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35348/0 (socket says 172.21.15.7:35348) 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2837996512 0 0) 0x7fd6fc1340f0 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): my global_id is 95663 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6dc003520 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535198285 0 0) 0x7fd6fc0a0f60 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): my global_id is 124179 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6fc1340f0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1305567180 0 0) 0x7fd6fc1359b0 con 0x7fd6fc137940 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): my global_id is 76458 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6fc0a0f60 con 0x7fd6fc137940 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 264276969 0 0) 0x7fd6fc1340f0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6fc1359b0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6f40019b0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): dump: 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.310 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2890726669 0 0) 0x7fd6dc003520 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6fc1340f0 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6e40014d0 con 0x7fd6fc0a49a0 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 10 monclient(hunting): dump: 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.311 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1906155854 0 0) 0x7fd6fc1359b0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.297+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.81:6790/0 conn(0x7fd6fc137940 legacy=0x7fd6fc1d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.7:6790/0 conn(0x7fd6fc0a49a0 legacy=0x7fd6fc132500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 1 monclient: found mon.d 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6fc0a1f30 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 5 monclient: authenticate success, global_id 124179 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 10 monclient: _renew_subs 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:28.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd6fc1332e0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 10 monclient: _renew_subs 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fd6fc137640 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd6f40038c0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6f4003e30 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6faffd640 4 set_mon_vals no callback set 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 10 monclient: dump: 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.314 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd6f4003fd0 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.298+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd6e00065c0 con 0x7fd6e0002860 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(294..294 src has 1..294) ==== 3898+0+0 (unknown 1267952350 0 0) 0x7fd6f4046d20 con 0x7fd6fc0a41b0 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd701312640 10 monclient: get_auth_request con 0x7fd6dc039fb0 auth_method 0 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029137427 0 0) 0x7fd6e00065c0 con 0x7fd6e0002860 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 10 monclient(hunting): my global_id is 76467 2024-05-08T15:53:28.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6fc1359b0 con 0x7fd6e0002860 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1149413275 0 0) 0x7fd6fc1359b0 con 0x7fd6e0002860 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6e00065c0 con 0x7fd6e0002860 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6e4004ba0 con 0x7fd6e0002860 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 10 monclient: dump: 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.317 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2874786996 0 0) 0x7fd6e00065c0 con 0x7fd6e0002860 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.299+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd6e000a150 con 0x7fd6e0002860 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.300+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fd6e4004ed0 con 0x7fd6e0002860 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.300+0000 7fd6f97fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.300+0000 7fd6f97fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:28.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.300+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.81:6790/0 conn(0x7fd6e0002860 legacy=0x7fd6e0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.300+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.81:6790/0 conn(0x7fd6e0002860 legacy=0x7fd6e0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.306+0000 7fd702d75640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.306+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd6e0009e40 con 0x7fd6e0009100 2024-05-08T15:53:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.306+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2132446111 0 0) 0x7fd6e0009e40 con 0x7fd6e0009100 2024-05-08T15:53:28.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient(hunting): my global_id is 76476 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6e00065c0 con 0x7fd6e0009100 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3699628696 0 0) 0x7fd6e00065c0 con 0x7fd6e0009100 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd6e0009e40 con 0x7fd6e0009100 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd6e4005740 con 0x7fd6e0009100 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient: dump: 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.321 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2897351094 0 0) 0x7fd6e0009e40 con 0x7fd6e0009100 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd6e000c3d0 con 0x7fd6e0009100 2024-05-08T15:53:28.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 2134841968) 0x7fd6e4006b60 con 0x7fd6e0009100 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.81:6790/0 conn(0x7fd6e0009100 legacy=0x7fd6e0009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.307+0000 7fd6f97fa640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.81:6790/0 conn(0x7fd6e0009100 legacy=0x7fd6e0009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 400597, 2024-05-08T15:53:28.323 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:28.324 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:28.325 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:28.327 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.328 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.329 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.330 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.331 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:28.333 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:28.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.336 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:28.338 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:28.342 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.81:6812/847627118 conn(0x7fd6dc039fb0 legacy=0x7fd6dc03c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 10 monclient: shutdown 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 >> v1:172.21.15.7:6792/0 conn(0x7fd6fc0a41b0 legacy=0x7fd6fc131df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd701312640 1 -- 172.21.15.7:0/4228753062 reap_dead start 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd701312640 1 -- 172.21.15.7:0/4228753062 reap_dead start 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 shutdown_connections 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 >> 172.21.15.7:0/4228753062 conn(0x7fd6fc0a9fa0 msgr2=0x7fd6fc130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 shutdown_connections 2024-05-08T15:53:28.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.308+0000 7fd702d75640 1 -- 172.21.15.7:0/4228753062 wait complete. 2024-05-08T15:53:28.345 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:28.345 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:53:28.345 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:53:28.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.452+0000 7fbcecdb6640 10 monclient: get_monmap_and_config 2024-05-08T15:53:28.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.452+0000 7fbcecdb6640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.452+0000 7fbcecdb6640 10 monclient: monmap: 2024-05-08T15:53:28.448 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.448 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.448 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:28.453524+0000 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:28.453524+0000 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.449 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 1 Processor -- start 2024-05-08T15:53:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 1 -- start start 2024-05-08T15:53:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient: init 2024-05-08T15:53:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient: _add_conns ranks=[3,8,1,5,0,7,4,2,6] 2024-05-08T15:53:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient(hunting): picked mon.noname-g con 0x7fbce8137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:28.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient(hunting): picked mon.noname-e con 0x7fbce80a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient(hunting): picked mon.noname-f con 0x7fbce80a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce80a2c70 con 0x7fbce80a41b0 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce80a2f50 con 0x7fbce8137940 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce80a1670 con 0x7fbce80a49a0 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcecdb6640 10 monclient(hunting): authenticate will time out at 1216.568457s 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcdffff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fbce80a49a0 0x7fbce813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35568/0 (socket says 172.21.15.7:35568) 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.453+0000 7fbcdffff640 1 -- 172.21.15.7:0/1758424416 learned_addr learned my addr 172.21.15.7:0/1758424416 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093049215 0 0) 0x7fbce80a1670 con 0x7fbce80a49a0 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient(hunting): my global_id is 95660 2024-05-08T15:53:28.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcd4003520 con 0x7fbce80a49a0 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2962319264 0 0) 0x7fbcd4003520 con 0x7fbce80a49a0 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 >> v1:172.21.15.81:6790/0 conn(0x7fbce8137940 legacy=0x7fbce8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 >> v1:172.21.15.81:6789/0 conn(0x7fbce80a41b0 legacy=0x7fbce8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 monclient: found mon.noname-e 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce80a39f0 con 0x7fbce80a49a0 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbcd00017b0 con 0x7fbce80a49a0 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbcecdb6640 5 monclient: authenticate success, global_id 95660 2024-05-08T15:53:28.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbcecdb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: dump: 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.461 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbcecdb6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbcd0001aa0 con 0x7fbce80a49a0 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 1 -- 172.21.15.7:0/1758424416 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbcd0004be0 con 0x7fbce80a49a0 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbcecdb6640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbcecdb6640 4 set_mon_vals no callback set 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.454+0000 7fbce5d9b640 10 monclient: dump: 2024-05-08T15:53:28.462 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.463 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 10 monclient: shutdown 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 1 -- 172.21.15.7:0/1758424416 >> v1:172.21.15.7:6793/0 conn(0x7fbce80a49a0 legacy=0x7fbce813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 1 -- 172.21.15.7:0/1758424416 shutdown_connections 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 1 -- 172.21.15.7:0/1758424416 >> 172.21.15.7:0/1758424416 conn(0x7fbce80a9fa0 msgr2=0x7fbce80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 1 -- 172.21.15.7:0/1758424416 shutdown_connections 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 1 -- 172.21.15.7:0/1758424416 wait complete. 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.455+0000 7fbcecdb6640 10 monclient: monmap: 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:28.464 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 1 Processor -- start 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 1 -- start start 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient: init 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient: _add_conns ranks=[8,2,6,0,7,3,5,1,4] 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient(hunting): picked mon.noname-e con 0x7fbce80a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient(hunting): picked mon.noname-b con 0x7fbce80a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient(hunting): picked mon.noname-d con 0x7fbce8137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce8137770 con 0x7fbce80a49a0 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce80a0f60 con 0x7fbce8137940 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbce8134dd0 con 0x7fbce80a41b0 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbcecdb6640 10 monclient(hunting): authenticate will time out at 1216.571497s 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbce659c640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fbce80a41b0 0x7fbce8137060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35576/0 (socket says 172.21.15.7:35576) 2024-05-08T15:53:28.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.456+0000 7fbce659c640 1 -- 172.21.15.7:0/205373735 learned_addr learned my addr 172.21.15.7:0/205373735 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3970904676 0 0) 0x7fbce8134dd0 con 0x7fbce80a41b0 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): my global_id is 95669 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbcc8003520 con 0x7fbce80a41b0 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3327804207 0 0) 0x7fbce8137770 con 0x7fbce80a49a0 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): my global_id is 95672 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce8134dd0 con 0x7fbce80a49a0 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1369850391 0 0) 0x7fbce80a0f60 con 0x7fbce8137940 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): my global_id is 124188 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce8137770 con 0x7fbce8137940 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2034226843 0 0) 0x7fbcc8003520 con 0x7fbce80a41b0 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbce80a0f60 con 0x7fbce80a41b0 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbcd8001a30 con 0x7fbce80a41b0 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): dump: 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.468 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3916953701 0 0) 0x7fbce8134dd0 con 0x7fbce80a49a0 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbcc8003520 con 0x7fbce80a49a0 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3560722467 0 0) 0x7fbce8137770 con 0x7fbce8137940 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbce8134dd0 con 0x7fbce8137940 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbcd0001770 con 0x7fbce80a49a0 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): dump: 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.470 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbce0001c00 con 0x7fbce8137940 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient(hunting): dump: 2024-05-08T15:53:28.471 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.472 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1657810370 0 0) 0x7fbce80a0f60 con 0x7fbce80a41b0 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.7:6792/0 conn(0x7fbce8137940 legacy=0x7fbce81d4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.7:6790/0 conn(0x7fbce80a49a0 legacy=0x7fbce81d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 monclient: found mon.e 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce8135b30 con 0x7fbce80a41b0 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcecdb6640 5 monclient: authenticate success, global_id 95669 2024-05-08T15:53:28.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcecdb6640 10 monclient: _renew_subs 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcecdb6640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fbce8135290 con 0x7fbce80a41b0 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcecdb6640 10 monclient: _renew_subs 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcecdb6640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fbce8134330 con 0x7fbce80a41b0 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fbcd8001d30 con 0x7fbce80a41b0 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbcd8003c00 con 0x7fbce80a41b0 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbce4d99640 4 set_mon_vals no callback set 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.457+0000 7fbcdeffd640 10 monclient: dump: 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.474 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.475 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.458+0000 7fbcecdb6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.458+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fbcd8003da0 con 0x7fbce80a41b0 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.458+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbccc0065c0 con 0x7fbccc002860 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.458+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(294..294 src has 1..294) ==== 3898+0+0 (unknown 1267952350 0 0) 0x7fbcd8046d70 con 0x7fbce80a41b0 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbce6d9d640 10 monclient: get_auth_request con 0x7fbcc803a4e0 auth_method 0 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554231106 0 0) 0x7fbccc0065c0 con 0x7fbccc002860 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 10 monclient(hunting): my global_id is 6341 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbce80a0f60 con 0x7fbccc002860 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2177598468 0 0) 0x7fbce80a0f60 con 0x7fbccc002860 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbccc0065c0 con 0x7fbccc002860 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbcd0005230 con 0x7fbccc002860 2024-05-08T15:53:28.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 10 monclient: dump: 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.477 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1057486693 0 0) 0x7fbccc0065c0 con 0x7fbccc002860 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.459+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbccc00a150 con 0x7fbccc002860 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.460+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fbcd0005360 con 0x7fbccc002860 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.460+0000 7fbcdeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.460+0000 7fbcdeffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.460+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.81:6791/0 conn(0x7fbccc002860 legacy=0x7fbccc004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.460+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.81:6791/0 conn(0x7fbccc002860 legacy=0x7fbccc004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.466+0000 7fbcecdb6640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.466+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fbccc009e40 con 0x7fbccc009100 2024-05-08T15:53:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1278175355 0 0) 0x7fbccc009e40 con 0x7fbccc009100 2024-05-08T15:53:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 10 monclient(hunting): my global_id is 6350 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbccc0065c0 con 0x7fbccc009100 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3180991461 0 0) 0x7fbccc0065c0 con 0x7fbccc009100 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fbccc009e40 con 0x7fbccc009100 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fbcd0005b40 con 0x7fbccc009100 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 10 monclient: dump: 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.481 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4219750910 0 0) 0x7fbccc009e40 con 0x7fbccc009100 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbccc00c3d0 con 0x7fbccc009100 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.467+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 159141126) 0x7fbcd0007fb0 con 0x7fbccc009100 2024-05-08T15:53:28.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcdeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcdeffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.81:6791/0 conn(0x7fbccc009100 legacy=0x7fbccc009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcdeffd640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.81:6791/0 conn(0x7fbccc009100 legacy=0x7fbccc009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 400773, 2024-05-08T15:53:28.483 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:28.484 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:28.485 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:28.486 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:28.487 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.488 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:28.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.492 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.493 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:28.497 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.498 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.499 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.500 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.501 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:28.501 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.81:6812/847627118 conn(0x7fbcc803a4e0 legacy=0x7fbcc803c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 10 monclient: shutdown 2024-05-08T15:53:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 >> v1:172.21.15.7:6793/0 conn(0x7fbce80a41b0 legacy=0x7fbce8137060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbce6d9d640 1 -- 172.21.15.7:0/205373735 reap_dead start 2024-05-08T15:53:28.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbce6d9d640 1 -- 172.21.15.7:0/205373735 reap_dead start 2024-05-08T15:53:28.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 shutdown_connections 2024-05-08T15:53:28.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 >> 172.21.15.7:0/205373735 conn(0x7fbce80a9fa0 msgr2=0x7fbce8130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 shutdown_connections 2024-05-08T15:53:28.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.468+0000 7fbcecdb6640 1 -- 172.21.15.7:0/205373735 wait complete. 2024-05-08T15:53:28.503 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:28.503 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:53:28.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:53:28.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.611+0000 7fced4dc7640 10 monclient: get_monmap_and_config 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.611+0000 7fced4dc7640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.611+0000 7fced4dc7640 10 monclient: monmap: 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:28.612725+0000 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:28.612725+0000 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.618 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.612+0000 7fced4dc7640 1 Processor -- start 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.612+0000 7fced4dc7640 1 -- start start 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.612+0000 7fced4dc7640 10 monclient: init 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 10 monclient: _add_conns ranks=[2,8,0,3,7,4,5,1,6] 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 10 monclient(hunting): picked mon.noname-b con 0x7fced0137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 10 monclient(hunting): picked mon.noname-e con 0x7fced00a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 10 monclient(hunting): picked mon.noname-a con 0x7fced00a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced00a2c70 con 0x7fced00a41b0 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced00a2f50 con 0x7fced0137940 2024-05-08T15:53:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced00a1670 con 0x7fced00a49a0 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcec7fff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fced00a49a0 0x7fced013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35578/0 (socket says 172.21.15.7:35578) 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 10 monclient(hunting): authenticate will time out at 1216.727843s 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcece59c640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fced0137940 0x7fced0137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51986/0 (socket says 172.21.15.7:51986) 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcec7fff640 1 -- 172.21.15.7:0/4215341633 learned_addr learned my addr 172.21.15.7:0/4215341633 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4176246220 0 0) 0x7fced00a1670 con 0x7fced00a49a0 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): my global_id is 95678 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcebc003520 con 0x7fced00a49a0 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454018752 0 0) 0x7fced00a2f50 con 0x7fced0137940 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): my global_id is 95681 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fced00a1670 con 0x7fced0137940 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1632087928 0 0) 0x7fced00a2c70 con 0x7fced00a41b0 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient(hunting): my global_id is 95571 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fced00a2f50 con 0x7fced00a41b0 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2689207198 0 0) 0x7fcebc003520 con 0x7fced00a49a0 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 >> v1:172.21.15.7:6790/0 conn(0x7fced0137940 legacy=0x7fced0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 >> v1:172.21.15.7:6789/0 conn(0x7fced00a41b0 legacy=0x7fced0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 monclient: found mon.noname-e 2024-05-08T15:53:28.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:53:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced00a39f0 con 0x7fced00a49a0 2024-05-08T15:53:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fceb80017b0 con 0x7fced00a49a0 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fcecdd9b640 10 monclient: dump: 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.623 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.624 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 5 monclient: authenticate success, global_id 95678 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.613+0000 7fced4dc7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fceb8001ad0 con 0x7fced00a49a0 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fcecdd9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fcecdd9b640 1 -- 172.21.15.7:0/4215341633 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fceb8001d40 con 0x7fced00a49a0 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fcecdd9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 4 set_mon_vals no callback set 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fcecdd9b640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fcecdd9b640 10 monclient: dump: 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.625 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 10 monclient: shutdown 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 1 -- 172.21.15.7:0/4215341633 >> v1:172.21.15.7:6793/0 conn(0x7fced00a49a0 legacy=0x7fced013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 1 -- 172.21.15.7:0/4215341633 shutdown_connections 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 1 -- 172.21.15.7:0/4215341633 >> 172.21.15.7:0/4215341633 conn(0x7fced00a9fa0 msgr2=0x7fced00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 1 -- 172.21.15.7:0/4215341633 shutdown_connections 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.614+0000 7fced4dc7640 1 -- 172.21.15.7:0/4215341633 wait complete. 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.615+0000 7fced4dc7640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.615+0000 7fced4dc7640 10 monclient: monmap: 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:28.627 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.615+0000 7fced4dc7640 1 Processor -- start 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.615+0000 7fced4dc7640 1 -- start start 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.615+0000 7fced4dc7640 10 monclient: init 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.615+0000 7fced4dc7640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.615+0000 7fced4dc7640 10 monclient: _add_conns ranks=[2,5,3,1,4,8,7,6,0] 2024-05-08T15:53:28.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient(hunting): picked mon.noname-b con 0x7fced00a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient(hunting): picked mon.noname-h con 0x7fced00a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient(hunting): picked mon.noname-g con 0x7fced0137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced01c9840 con 0x7fced00a41b0 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced0137720 con 0x7fced0137940 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fced0137160 con 0x7fced00a49a0 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient(hunting): authenticate will time out at 1216.730723s 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcece59c640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fced00a41b0 0x7fced01c9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51996/0 (socket says 172.21.15.7:51996) 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcece59c640 1 -- 172.21.15.7:0/1070062206 learned_addr learned my addr 172.21.15.7:0/1070062206 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776896423 0 0) 0x7fced01c9840 con 0x7fced00a41b0 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): my global_id is 95690 2024-05-08T15:53:28.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb0003520 con 0x7fced00a41b0 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2407891712 0 0) 0x7fceb0003520 con 0x7fced00a41b0 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fced01c9840 con 0x7fced00a41b0 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3992101755 0 0) 0x7fced0137160 con 0x7fced00a49a0 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): my global_id is 6359 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb0003520 con 0x7fced00a49a0 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1805176371 0 0) 0x7fced0137720 con 0x7fced0137940 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): my global_id is 76494 2024-05-08T15:53:28.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fced0137160 con 0x7fced0137940 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcec0001a90 con 0x7fced00a41b0 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.631 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2756702879 0 0) 0x7fced01c9840 con 0x7fced00a41b0 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.81:6791/0 conn(0x7fced00a49a0 legacy=0x7fced0141040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.81:6790/0 conn(0x7fced0137940 legacy=0x7fced01d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 monclient: found mon.b 2024-05-08T15:53:28.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced0137440 con 0x7fced00a41b0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 5 monclient: authenticate success, global_id 95690 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient: _renew_subs 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fced01366c0 con 0x7fced00a41b0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient: _renew_subs 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fced0135720 con 0x7fced00a41b0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcec0004a90 con 0x7fced00a41b0 2024-05-08T15:53:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcec0001d40 con 0x7fced00a41b0 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcecd59a640 4 set_mon_vals no callback set 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.616+0000 7fcec6ffd640 10 monclient: dump: 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.634 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.617+0000 7fced4dc7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.617+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fceb40065c0 con 0x7fceb4002860 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.617+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fcec0005000 con 0x7fced00a41b0 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.617+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(294..294 src has 1..294) ==== 3898+0+0 (unknown 1267952350 0 0) 0x7fcec0046fc0 con 0x7fced00a41b0 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2440244753 0 0) 0x7fceb40065c0 con 0x7fceb4002860 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient(hunting): my global_id is 114892 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fced01c9840 con 0x7fceb4002860 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fceced9d640 10 monclient: get_auth_request con 0x7fceb0039d30 auth_method 0 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 331072344 0 0) 0x7fced01c9840 con 0x7fceb4002860 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fceb40065c0 con 0x7fceb4002860 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fceb8001430 con 0x7fceb4002860 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient: dump: 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.636 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1588189306 0 0) 0x7fceb40065c0 con 0x7fceb4002860 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fceb400a150 con 0x7fceb4002860 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fceb80016e0 con 0x7fceb4002860 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:28.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.81:6792/0 conn(0x7fceb4002860 legacy=0x7fceb4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.618+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.81:6792/0 conn(0x7fceb4002860 legacy=0x7fceb4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fced4dc7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fceb4009e20 con 0x7fceb40090e0 2024-05-08T15:53:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1510890827 0 0) 0x7fceb4009e20 con 0x7fceb40090e0 2024-05-08T15:53:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fcec6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fcec6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fcec6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fcec6ffd640 10 monclient(hunting): my global_id is 114901 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.625+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fceb40065c0 con 0x7fceb40090e0 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3338467022 0 0) 0x7fceb40065c0 con 0x7fceb40090e0 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fceb4009e20 con 0x7fceb40090e0 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fceb8001430 con 0x7fceb40090e0 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 10 monclient: dump: 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.640 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3043320271 0 0) 0x7fceb4009e20 con 0x7fceb40090e0 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fceb400c390 con 0x7fceb40090e0 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1664912211) 0x7fceb8007630 con 0x7fceb40090e0 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:28.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.81:6792/0 conn(0x7fceb40090e0 legacy=0x7fceb4009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fcec6ffd640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.81:6792/0 conn(0x7fceb40090e0 legacy=0x7fceb4009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 143491, 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:28.642 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.643 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.644 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:28.645 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:28.646 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:28.647 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:28.649 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.650 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.651 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.652 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:28.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.656 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:28.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:28.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.660 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:28.661 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:53:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:28.662 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:28.662 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:28.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.626+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.81:6812/847627118 conn(0x7fceb0039d30 legacy=0x7fceb003c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fced4dc7640 10 monclient: shutdown 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fced4dc7640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 >> v1:172.21.15.7:6790/0 conn(0x7fced00a41b0 legacy=0x7fced01c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fceced9d640 1 -- 172.21.15.7:0/1070062206 reap_dead start 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fceced9d640 1 -- 172.21.15.7:0/1070062206 reap_dead start 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 shutdown_connections 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 >> 172.21.15.7:0/1070062206 conn(0x7fced00a9fa0 msgr2=0x7fced01307d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 shutdown_connections 2024-05-08T15:53:28.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.627+0000 7fced4dc7640 1 -- 172.21.15.7:0/1070062206 wait complete. 2024-05-08T15:53:28.664 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:28.664 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T15:53:28.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T15:53:28.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.772+0000 7f7044585640 10 monclient: get_monmap_and_config 2024-05-08T15:53:28.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.772+0000 7f7044585640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.772+0000 7f7044585640 10 monclient: monmap: 2024-05-08T15:53:28.768 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.768 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:28.773495+0000 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:28.773495+0000 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.769 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 1 Processor -- start 2024-05-08T15:53:28.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 1 -- start start 2024-05-08T15:53:28.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient: init 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient: _add_conns ranks=[3,1,2,5,0,8,6,4,7] 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient(hunting): picked mon.noname-g con 0x7f703c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient(hunting): picked mon.noname-f con 0x7f703c0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient(hunting): picked mon.noname-b con 0x7f703c0a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c0a2c70 con 0x7f703c0a49a0 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c0a2f50 con 0x7f703c0a41b0 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c0a1670 con 0x7f703c137940 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7044585640 10 monclient(hunting): authenticate will time out at 1216.888464s 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7042b22640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f703c0a41b0 0x7f703c140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:52002/0 (socket says 172.21.15.7:52002) 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.773+0000 7f7042b22640 1 -- 172.21.15.7:0/4035208199 learned_addr learned my addr 172.21.15.7:0/4035208199 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2718035830 0 0) 0x7f703c0a2f50 con 0x7f703c0a41b0 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): my global_id is 95699 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7028003520 con 0x7f703c0a41b0 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1853613555 0 0) 0x7f703c0a2c70 con 0x7f703c0a49a0 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): my global_id is 6148 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f703c0a2f50 con 0x7f703c0a49a0 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824326841 0 0) 0x7f703c0a1670 con 0x7f703c137940 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient(hunting): my global_id is 76503 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f703c0a2c70 con 0x7f703c137940 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 135613055 0 0) 0x7f7028003520 con 0x7f703c0a41b0 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 >> v1:172.21.15.81:6790/0 conn(0x7f703c137940 legacy=0x7f703c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 >> v1:172.21.15.81:6789/0 conn(0x7f703c0a49a0 legacy=0x7f703c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 monclient: found mon.noname-b 2024-05-08T15:53:28.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c0a39f0 con 0x7f703c0a41b0 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7034001b30 con 0x7f703c0a41b0 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f7044585640 5 monclient: authenticate success, global_id 95699 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f7044585640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: dump: 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.782 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f7044585640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7034001e50 con 0x7f703c0a41b0 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 1 -- 172.21.15.7:0/4035208199 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7034004f60 con 0x7f703c0a41b0 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f7044585640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f7044585640 4 set_mon_vals no callback set 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.774+0000 7f704131f640 10 monclient: dump: 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.784 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.785 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.785 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.785 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 10 monclient: shutdown 2024-05-08T15:53:28.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 1 -- 172.21.15.7:0/4035208199 >> v1:172.21.15.7:6790/0 conn(0x7f703c0a41b0 legacy=0x7f703c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 1 -- 172.21.15.7:0/4035208199 shutdown_connections 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 1 -- 172.21.15.7:0/4035208199 >> 172.21.15.7:0/4035208199 conn(0x7f703c0a9fa0 msgr2=0x7f703c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 1 -- 172.21.15.7:0/4035208199 shutdown_connections 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 1 -- 172.21.15.7:0/4035208199 wait complete. 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 10 monclient: build_initial_monmap 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.775+0000 7f7044585640 10 monclient: monmap: 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:28.786 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:28.787 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 1 Processor -- start 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 1 -- start start 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient: init 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient: _add_conns ranks=[2,4,0,6,5,1,8,3,7] 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient(hunting): picked mon.noname-b con 0x7f703c0a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient(hunting): picked mon.noname-c con 0x7f703c0a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient(hunting): picked mon.noname-a con 0x7f703c137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c1340f0 con 0x7f703c137940 2024-05-08T15:53:28.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c0a1f30 con 0x7f703c0a41b0 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f703c0a0f60 con 0x7f703c0a49a0 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7044585640 10 monclient(hunting): authenticate will time out at 1216.891198s 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7042321640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f703c0a41b0 0x7f703c1321c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:52008/0 (socket says 172.21.15.7:52008) 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7042321640 1 -- 172.21.15.7:0/563007702 learned_addr learned my addr 172.21.15.7:0/563007702 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4237673564 0 0) 0x7f703c0a1f30 con 0x7f703c0a41b0 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): my global_id is 95708 2024-05-08T15:53:28.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7020003520 con 0x7f703c0a41b0 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2642896301 0 0) 0x7f703c0a0f60 con 0x7f703c0a49a0 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): my global_id is 124159 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f703c0a1f30 con 0x7f703c0a49a0 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313428530 0 0) 0x7f703c1340f0 con 0x7f703c137940 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:28.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 10 monclient(hunting): my global_id is 95580 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.776+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f703c0a0f60 con 0x7f703c137940 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2451726724 0 0) 0x7f7020003520 con 0x7f703c0a41b0 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f703c1340f0 con 0x7f703c0a41b0 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7038001a90 con 0x7f703c0a41b0 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:28.791 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1731186905 0 0) 0x7f703c0a1f30 con 0x7f703c0a49a0 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7020003520 con 0x7f703c0a49a0 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3279958284 0 0) 0x7f703c0a0f60 con 0x7f703c137940 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f703c0a1f30 con 0x7f703c137940 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f702c001a50 con 0x7f703c0a49a0 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.793 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7034004ea0 con 0x7f703c137940 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:28.794 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.795 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3179973990 0 0) 0x7f703c1340f0 con 0x7f703c0a41b0 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 >> v1:172.21.15.7:6791/0 conn(0x7f703c0a49a0 legacy=0x7f703c1328d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 >> v1:172.21.15.7:6789/0 conn(0x7f703c137940 legacy=0x7f703c1d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 monclient: found mon.b 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c133a10 con 0x7f703c0a41b0 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7044585640 5 monclient: authenticate success, global_id 95708 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7044585640 10 monclient: _renew_subs 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7044585640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f703c132fe0 con 0x7f703c0a41b0 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7044585640 10 monclient: _renew_subs 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7044585640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f703c135d70 con 0x7f703c0a41b0 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f70380038c0 con 0x7f703c0a41b0 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7038003e30 con 0x7f703c0a41b0 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7040b1e640 4 set_mon_vals no callback set 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.777+0000 7f7032ffd640 10 monclient: dump: 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:28.797 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.778+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7038003fd0 con 0x7f703c0a41b0 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.778+0000 7f7044585640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.778+0000 7f7044585640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.778+0000 7f7044585640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:28.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.778+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7024005080 con 0x7f703c0a41b0 2024-05-08T15:53:28.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.779+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(294..294 src has 1..294) ==== 3898+0+0 (unknown 1267952350 0 0) 0x7f7038046db0 con 0x7f703c0a41b0 2024-05-08T15:53:28.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.779+0000 7f7041b20640 10 monclient: get_auth_request con 0x7f702003a4f0 auth_method 0 2024-05-08T15:53:28.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.787+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f7038015e20 con 0x7f703c0a41b0 2024-05-08T15:53:28.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.787+0000 7f7032ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:28.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:28.787+0000 7f7032ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:28.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38870]: starting. 2024-05-08T15:53:28.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38870]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38868 2024-05-08T15:53:28.800 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38870]: shutting down. 2024-05-08T15:53:29.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.052+0000 7f7044585640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-05-08T15:53:29.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.052+0000 7f7044585640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-05-08T15:53:29.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.052+0000 7f7044585640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:29.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.052+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f7024005b80 con 0x7f703c0a41b0 2024-05-08T15:53:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.055+0000 7f7032ffd640 1 -- 172.21.15.7:0/563007702 <== mon.2 v1:172.21.15.7:6790/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f703801e2d0 con 0x7f703c0a41b0 2024-05-08T15:53:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.055+0000 7f7032ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-05-08T15:53:29.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.055+0000 7f7032ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:29.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.057+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 >> v1:172.21.15.81:6812/847627118 conn(0x7f702003a4f0 legacy=0x7f702003c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:29.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.057+0000 7f7044585640 10 monclient: shutdown 2024-05-08T15:53:29.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.057+0000 7f7044585640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:29.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.057+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 >> v1:172.21.15.7:6790/0 conn(0x7f703c0a41b0 legacy=0x7f703c1321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:29.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.057+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 shutdown_connections 2024-05-08T15:53:29.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.058+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 >> 172.21.15.7:0/563007702 conn(0x7f703c0a9fa0 msgr2=0x7f703c130c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:29.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.058+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 shutdown_connections 2024-05-08T15:53:29.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:29.058+0000 7f7044585640 1 -- 172.21.15.7:0/563007702 wait complete. 2024-05-08T15:53:29.072 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-05-08T15:53:29.120 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.120 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.120 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=4178395373,config=1618278320,health=796319130,kv=2767944899,logm=1706928915}) 2024-05-08T15:53:29.120 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.120 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=20,config=1,health=12,kv=9,logm=58} crc {auth=4178395373,config=2457401934,health=274813237,kv=313657359,logm=1720864728}) 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=19,config=2,health=11,kv=10,logm=58} crc {auth=1683316103,config=1618278320,health=1830930674,kv=3560234957,logm=1578065529}) 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=18,config=1,health=11,kv=10,logm=60} crc {auth=4269760382,config=1228700967,health=1830930674,kv=3755490522,logm=434535848}) 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.121 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=19,config=1,health=11,kv=8,logm=61} crc {auth=344690608,config=1228700967,health=2805248589,kv=430915584,logm=1004245422}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=19,config=2,health=12,kv=9,logm=58} crc {auth=1683316104,config=1618278320,health=274813237,kv=368437079,logm=3144253280}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=20,config=2,health=11,kv=9,logm=58} crc {auth=1334162626,config=1618278320,health=11389592,kv=313657359,logm=4249459284}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=19,config=1,health=12,kv=10,logm=58} crc {auth=1646254262,config=1228700967,health=1926008639,kv=3560234957,logm=733126282}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.055+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=19,config=2,health=11,kv=9,logm=59} crc {auth=2373903812,config=3625383534,health=4038449461,kv=2767944899,logm=2602991235}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3142198795}) 2024-05-08T15:53:29.122 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1723525391}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2178820509}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3414659606}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2045552406}) 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.123 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2613622356}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3257541391}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.056+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.057+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3178854750}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.057+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3699774100}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=996811297}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3308933532}) 2024-05-08T15:53:29.124 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4040899567}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=600946124}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=99446374}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2177356699}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.125 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4130684052}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2160978382}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.058+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=966706711}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1435041975}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1006893428}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=876010279}) 2024-05-08T15:53:29.126 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2371533803}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4266657852}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1890176312}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1191429005}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3215739495}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.059+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4029835927}) 2024-05-08T15:53:29.127 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3212312398}) 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2828270212}) 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1612129471}) 2024-05-08T15:53:29.128 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3329567822}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3885201639}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1403278447}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1105570690}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=23059018}) 2024-05-08T15:53:29.129 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.061+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2260666447}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3302396976}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4256834392}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3828984999}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=363366943}) 2024-05-08T15:53:29.130 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3825255923}) 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3642666270}) 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1657609886}) 2024-05-08T15:53:29.131 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3470400729}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.062+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1357083344}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1384212388}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=508116476}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.063+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2813096270}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1921127211}) 2024-05-08T15:53:29.132 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=528340909}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4091874994}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=600025118}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=98197632}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.064+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3195609090}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.133 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2597477157}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2317886103}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4099998284}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3424009951}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3407460186}) 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.134 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=184754131}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4063658318}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4213364977}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.065+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3797028178}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=127076085}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=494104472}) 2024-05-08T15:53:29.135 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1640076716}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=324934735}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.066+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3680887012}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2341239508}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3419533360}) 2024-05-08T15:53:29.136 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4254095501}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.067+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1206784965}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4120645164}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3531615026}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=792756354}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.137 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=877801986}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2654595929}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2324742172}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=631971950}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2603203737}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.068+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1033352569}) 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.138 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2299804550}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1044364935}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1894671546}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3135697449}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1118564474}) 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.139 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.069+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3089848352}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.070+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.070+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.070+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=668675663}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.070+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.070+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2196248174}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.070+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=149026958}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2456527764}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4081307432}) 2024-05-08T15:53:29.140 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1154335033}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1950810564}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1352332819}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3691644856}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.141 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=747264139}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1498957478}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.071+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=544864062}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4040549341}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=766660256}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3305288710}) 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.142 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2254002295}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2594503124}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2101849252}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3963801071}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3347371973}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.072+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2963686266}) 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.073+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.143 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.073+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.073+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3885381373}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.073+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1860272199}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=333361430}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2196782828}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4275092401}) 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.144 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=54466274}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=151377638}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1088247885}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.074+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3119969045}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2773162107}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1793350466}) 2024-05-08T15:53:29.145 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2656711430}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3392910657}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1423872143}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2807026143}) 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.146 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2665068923}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1126319775}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.075+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1466365527}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4142312653}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4244551920}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.147 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3458634231}) 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.076+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2185473299}) 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3878605693}) 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2540965187}) 2024-05-08T15:53:29.148 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2257388408}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=862068584}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.077+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=63603011}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2723937292}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1426858578}) 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.149 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1496489179}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2457479534}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3567096071}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2766027332}) 2024-05-08T15:53:29.150 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=745717864}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1441324354}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.078+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2560167191}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3290294644}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2356610576}) 2024-05-08T15:53:29.151 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1632740459}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2055521701}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.079+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=806796484}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3495478285}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=84099553}) 2024-05-08T15:53:29.152 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3913088429}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.080+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=434555765}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3206392419}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1776102519}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1752986963}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.153 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1915833076}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1786846474}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1223987677}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1567030069}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3131245715}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.081+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3749713482}) 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.154 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2603493608}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1524307627}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=959413523}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3413324027}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3123148590}) 2024-05-08T15:53:29.155 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=987036880}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1877902020}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.082+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2907775165}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.083+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3046039494}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1265275159}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.156 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3372660505}) 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=681396803}) 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=63869046}) 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.157 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2294599812}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2976313090}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1241435321}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1519611695}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.084+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2861874904}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.085+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=70,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=10,mon_config_key=8} crc {logm=46634338,mdsmap=2840952586,mgr=584111586,mgr_metadata=1815193447,mgrstat=420333903,mon_config_key=619004784}) 2024-05-08T15:53:29.158 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=74,mdsmap=2,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=8,mon_config_key=6} crc {logm=1257418034,mdsmap=3140514009,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=4034063713,mon_config_key=3885491317}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=71,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=6,monmap=2} crc {logm=2886207261,mdsmap=2840952586,mgr=3794531988,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=420333903,mon_config_key=3387683994,monmap=654611407}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=73,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=5} crc {logm=2103288356,mdsmap=2091496210,mgr=649407495,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=420333903,mon_config_key=1741757882}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=73,mdsmap=2,mgr=7,mgr_metadata=1,mgrstat=9,mon_config_key=8} crc {logm=810035849,mdsmap=1401145748,mgr=385272302,mgr_metadata=1815193447,mgrstat=2420930409,mon_config_key=619004784}) 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.159 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=73,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=4} crc {logm=3969947645,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=420333903,mon_config_key=1994892838}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=69,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=1} crc {logm=764731059,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=861967047,mon_config_key=619004784,monmap=2540908388}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=6} crc {logm=4123281490,mdsmap=2840952586,mgr=584111586,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1862064578,mon_config_key=3885491317}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.086+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=72,mdsmap=3,mgr=8,mgr_metadata=1,mgrstat=10,mon_config_key=6} crc {logm=169565241,mdsmap=2840952586,mgr=584111586,mgr_metadata=1815193447,mgrstat=420333903,mon_config_key=3885491317}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {mon_config_key=1,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=3597491434,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=1495227510}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.160 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {mon_config_key=2,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=88} crc {mon_config_key=1285499322,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=3490237693}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {mon_config_key=1,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=90} crc {mon_config_key=3597491434,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=1497673100}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {mon_config_key=2,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=88} crc {mon_config_key=1285499322,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=4132007603}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2266874790}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {mon_config_key=1,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=90} crc {mon_config_key=3597491434,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=982271577}) 2024-05-08T15:53:29.161 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.087+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.088+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.088+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {mon_config_key=2,monmap=3,osd_metadata=5,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=256126145,osd_metadata=3047975888,osd_pg_creating=2528475529,osdmap=2628290688}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.088+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.088+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {mon_config_key=2,monmap=2,osd_metadata=6,osd_pg_creating=1,osdmap=89} crc {mon_config_key=1285499322,monmap=654611407,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=2566419321}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.088+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {mon_config_key=2,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=88} crc {mon_config_key=1285499322,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2528475529,osdmap=541449123}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=212927060}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2375819959}) 2024-05-08T15:53:29.162 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2808570734}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3072677771}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3725519988}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=885946975}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.163 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=52520818}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1074112557}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.089+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=3069769802}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1057366784}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4013823250}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3115739574}) 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.164 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=3803426947}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=1195512105}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1760600713}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1599329280}) 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.165 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=791223336}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.091+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=1826817323}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=1916260844}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=4280992204}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1393629325}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=4213300941}) 2024-05-08T15:53:29.166 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3736685860}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=1857574965}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=1615194760}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=1467396258}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.093+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=311604846}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.094+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.094+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=45} crc {osdmap=616944881}) 2024-05-08T15:53:29.167 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=47} crc {osdmap=4193250101}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=48} crc {osdmap=2375366442}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=45} crc {osdmap=2813588686}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=48} crc {osdmap=1163849541}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.168 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=48} crc {osdmap=3381605085}) 2024-05-08T15:53:29.169 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.169 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.169 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=50} crc {osdmap=1730342305}) 2024-05-08T15:53:29.169 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:53:29.169 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=49} crc {osdmap=686015087}) 2024-05-08T15:53:29.169 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:53:29.095+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=51} crc {osdmap=2592127838}) 2024-05-08T15:53:30.073 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:53:30.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:53:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.192+0000 7f661eca5640 10 monclient: get_monmap_and_config 2024-05-08T15:53:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.192+0000 7f661eca5640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.192+0000 7f661eca5640 10 monclient: monmap: 2024-05-08T15:53:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:30.193504+0000 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:30.193504+0000 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.189 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 1 Processor -- start 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 1 -- start start 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient: init 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient: _add_conns ranks=[0,6,4,2,5,3,1,7,8] 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient(hunting): picked mon.noname-a con 0x7f6618137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient(hunting): picked mon.noname-d con 0x7f66180a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient(hunting): picked mon.noname-c con 0x7f66180a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66180a2c70 con 0x7f6618137940 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66180a2f50 con 0x7f66180a41b0 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66180a1670 con 0x7f66180a49a0 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.193+0000 7f661eca5640 10 monclient(hunting): authenticate will time out at 1218.308554s 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f661d242640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f66180a41b0 0x7f6618140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51864/0 (socket says 172.21.15.7:51864) 2024-05-08T15:53:30.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f661ca41640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f6618137940 0x7f6618137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39114/0 (socket says 172.21.15.7:39114) 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f661d242640 1 -- 172.21.15.7:0/39469057 learned_addr learned my addr 172.21.15.7:0/39469057 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660ffff640 1 --1- 172.21.15.7:0/39469057 >> v1:172.21.15.7:6792/0 conn(0x7f66180a49a0 0x7f661813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35356/0 (socket says 172.21.15.7:35356) 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2631466997 0 0) 0x7f66180a2c70 con 0x7f6618137940 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): my global_id is 95616 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6600003520 con 0x7f6618137940 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2749542084 0 0) 0x7f66180a2f50 con 0x7f66180a41b0 2024-05-08T15:53:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): my global_id is 124168 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66180a2c70 con 0x7f66180a41b0 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1819479 0 0) 0x7f66180a1670 con 0x7f66180a49a0 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient(hunting): my global_id is 124215 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66180a2f50 con 0x7f66180a49a0 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1850652062 0 0) 0x7f6600003520 con 0x7f6618137940 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 >> v1:172.21.15.7:6792/0 conn(0x7f66180a49a0 legacy=0x7f661813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 >> v1:172.21.15.7:6791/0 conn(0x7f66180a41b0 legacy=0x7f6618140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 monclient: found mon.noname-a 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66180a39f0 con 0x7f6618137940 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f66140016e0 con 0x7f6618137940 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f661eca5640 5 monclient: authenticate success, global_id 95616 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: dump: 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.200 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f661eca5640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6614001a00 con 0x7f6618137940 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 1 -- 172.21.15.7:0/39469057 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6614004b20 con 0x7f6618137940 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f660f7fe640 10 monclient: dump: 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f661eca5640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.194+0000 7f661eca5640 4 set_mon_vals no callback set 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.195+0000 7f661eca5640 10 monclient: shutdown 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.195+0000 7f661eca5640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.195+0000 7f661eca5640 1 -- 172.21.15.7:0/39469057 >> v1:172.21.15.7:6789/0 conn(0x7f6618137940 legacy=0x7f6618137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.195+0000 7f661eca5640 1 -- 172.21.15.7:0/39469057 shutdown_connections 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.195+0000 7f661eca5640 1 -- 172.21.15.7:0/39469057 >> 172.21.15.7:0/39469057 conn(0x7f66180a9fa0 msgr2=0x7f66180aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.195+0000 7f661eca5640 1 -- 172.21.15.7:0/39469057 shutdown_connections 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.195+0000 7f661eca5640 1 -- 172.21.15.7:0/39469057 wait complete. 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.196+0000 7f661eca5640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.196+0000 7f661eca5640 10 monclient: monmap: 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.196+0000 7f661eca5640 1 Processor -- start 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.196+0000 7f661eca5640 1 -- start start 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.196+0000 7f661eca5640 10 monclient: init 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 10 monclient: _add_conns ranks=[4,5,6,3,2,0,1,7,8] 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 10 monclient(hunting): picked mon.noname-c con 0x7f66180a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 10 monclient(hunting): picked mon.noname-h con 0x7f66180a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 10 monclient(hunting): picked mon.noname-d con 0x7f6618137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66181394b0 con 0x7f66180a41b0 2024-05-08T15:53:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66180a0f60 con 0x7f66180a49a0 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6618136930 con 0x7f6618137940 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661eca5640 10 monclient(hunting): authenticate will time out at 1218.311865s 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661ca41640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f66180a41b0 0x7f66181412e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51880/0 (socket says 172.21.15.7:51880) 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f661ca41640 1 -- 172.21.15.7:0/1271026984 learned_addr learned my addr 172.21.15.7:0/1271026984 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 317529158 0 0) 0x7f66180a39f0 con 0x7f66180a41b0 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): my global_id is 124177 2024-05-08T15:53:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6604003520 con 0x7f66180a41b0 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4234663130 0 0) 0x7f6618136930 con 0x7f6618137940 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): my global_id is 124224 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f66180a39f0 con 0x7f6618137940 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1254366371 0 0) 0x7f6604003520 con 0x7f66180a41b0 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6618136930 con 0x7f66180a41b0 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1356116453 0 0) 0x7f66180a0f60 con 0x7f66180a49a0 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): my global_id is 6377 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6604003520 con 0x7f66180a49a0 2024-05-08T15:53:30.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6614001c70 con 0x7f66180a41b0 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): dump: 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3529653298 0 0) 0x7f66180a39f0 con 0x7f6618137940 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f66180a0f60 con 0x7f6618137940 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f66100017d0 con 0x7f6618137940 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.197+0000 7f660d7fa640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient(hunting): dump: 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.211 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 628327903 0 0) 0x7f6618136930 con 0x7f66180a41b0 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 >> v1:172.21.15.7:6792/0 conn(0x7f6618137940 legacy=0x7f66181d4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 >> v1:172.21.15.81:6791/0 conn(0x7f66180a49a0 legacy=0x7f66181d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 1 monclient: found mon.c 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66181419f0 con 0x7f66180a41b0 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f661eca5640 5 monclient: authenticate success, global_id 124177 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f661eca5640 10 monclient: _renew_subs 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f661eca5640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f66181ccd20 con 0x7f66180a41b0 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f661eca5640 10 monclient: _renew_subs 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f661eca5640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f6618137670 con 0x7f66180a41b0 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6614004e40 con 0x7f66180a41b0 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f66140053b0 con 0x7f66180a41b0 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660effd640 4 set_mon_vals no callback set 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.198+0000 7f660d7fa640 10 monclient: dump: 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.213 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.214 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.199+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6614005550 con 0x7f66180a41b0 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.199+0000 7f661eca5640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.199+0000 7f661eca5640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.199+0000 7f661eca5640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.199+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f8005080 con 0x7f66180a41b0 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.199+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(296..296 src has 1..296) ==== 3898+0+0 (unknown 1741224607 0 0) 0x7f6614046ec0 con 0x7f66180a41b0 2024-05-08T15:53:30.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.199+0000 7f660ffff640 10 monclient: get_auth_request con 0x7f660403a230 auth_method 0 2024-05-08T15:53:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.204+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f66181394b0 con 0x7f66180a41b0 2024-05-08T15:53:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.204+0000 7f660d7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.204+0000 7f660d7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.463+0000 7f661eca5640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:53:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.463+0000 7f661eca5640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:53:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.463+0000 7f661eca5640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.463+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f65f8005370 con 0x7f66180a41b0 2024-05-08T15:53:30.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.463+0000 7f660d7fa640 1 -- 172.21.15.7:0/1271026984 <== mon.4 v1:172.21.15.7:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 1632057905) 0x7f6614015ff0 con 0x7f66180a41b0 2024-05-08T15:53:30.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.463+0000 7f660d7fa640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:53:30.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.463+0000 7f660d7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:30.460 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":50,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:53:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 >> v1:172.21.15.81:6812/847627118 conn(0x7f660403a230 legacy=0x7f660403c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 10 monclient: shutdown 2024-05-08T15:53:30.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 >> v1:172.21.15.7:6791/0 conn(0x7f66180a41b0 legacy=0x7f66181412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 shutdown_connections 2024-05-08T15:53:30.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 >> 172.21.15.7:0/1271026984 conn(0x7f66180a9fa0 msgr2=0x7f66180a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 shutdown_connections 2024-05-08T15:53:30.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.465+0000 7f661eca5640 1 -- 172.21.15.7:0/1271026984 wait complete. 2024-05-08T15:53:30.472 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:53:30.472 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-05-08T15:53:30.472 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:53:30.473 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:53:30.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.581+0000 7f79f7e4e640 10 monclient: get_monmap_and_config 2024-05-08T15:53:30.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.581+0000 7f79f7e4e640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.581+0000 7f79f7e4e640 10 monclient: monmap: 2024-05-08T15:53:30.577 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.577 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.577 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:30.582501+0000 2024-05-08T15:53:30.577 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:30.582501+0000 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.578 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 1 Processor -- start 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 1 -- start start 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient: init 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient: _add_conns ranks=[1,3,4,7,8,0,2,5,6] 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient(hunting): picked mon.noname-f con 0x7f79f0137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient(hunting): picked mon.noname-g con 0x7f79f00a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient(hunting): picked mon.noname-c con 0x7f79f00a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f00a2c70 con 0x7f79f0137940 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f00a2f50 con 0x7f79f00a49a0 2024-05-08T15:53:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f00a1670 con 0x7f79f00a41b0 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.582+0000 7f79f7e4e640 10 monclient(hunting): authenticate will time out at 1218.697528s 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f63eb640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f79f00a41b0 0x7f79f0140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51896/0 (socket says 172.21.15.7:51896) 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f63eb640 1 -- 172.21.15.7:0/4281067970 learned_addr learned my addr 172.21.15.7:0/4281067970 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2398335175 0 0) 0x7f79f00a1670 con 0x7f79f00a41b0 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): my global_id is 124186 2024-05-08T15:53:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79dc003520 con 0x7f79f00a41b0 2024-05-08T15:53:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344086421 0 0) 0x7f79f00a2f50 con 0x7f79f00a49a0 2024-05-08T15:53:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient(hunting): my global_id is 76539 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f00a1670 con 0x7f79f00a49a0 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3621680056 0 0) 0x7f79dc003520 con 0x7f79f00a41b0 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 >> v1:172.21.15.81:6790/0 conn(0x7f79f00a49a0 legacy=0x7f79f013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 >> v1:172.21.15.81:6789/0 conn(0x7f79f0137940 legacy=0x7f79f0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 monclient: found mon.noname-c 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f00a39f0 con 0x7f79f00a41b0 2024-05-08T15:53:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79e8001b30 con 0x7f79f00a41b0 2024-05-08T15:53:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: dump: 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.595 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.596 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f7e4e640 5 monclient: authenticate success, global_id 124186 2024-05-08T15:53:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f7e4e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f79e8001e50 con 0x7f79f00a41b0 2024-05-08T15:53:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 1 -- 172.21.15.7:0/4281067970 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79e80020c0 con 0x7f79f00a41b0 2024-05-08T15:53:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f7e4e640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f7e4e640 4 set_mon_vals no callback set 2024-05-08T15:53:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.583+0000 7f79f4be8640 10 monclient: dump: 2024-05-08T15:53:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.597 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.598 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.599 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 10 monclient: shutdown 2024-05-08T15:53:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 1 -- 172.21.15.7:0/4281067970 >> v1:172.21.15.7:6791/0 conn(0x7f79f00a41b0 legacy=0x7f79f0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 1 -- 172.21.15.7:0/4281067970 shutdown_connections 2024-05-08T15:53:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 1 -- 172.21.15.7:0/4281067970 >> 172.21.15.7:0/4281067970 conn(0x7f79f00a9fa0 msgr2=0x7f79f00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 1 -- 172.21.15.7:0/4281067970 shutdown_connections 2024-05-08T15:53:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 1 -- 172.21.15.7:0/4281067970 wait complete. 2024-05-08T15:53:30.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.584+0000 7f79f7e4e640 10 monclient: monmap: 2024-05-08T15:53:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:30.601 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.602 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 1 Processor -- start 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 1 -- start start 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient: init 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient: _add_conns ranks=[3,8,2,7,6,1,4,5,0] 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient(hunting): picked mon.noname-g con 0x7f79f00a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:30.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient(hunting): picked mon.noname-e con 0x7f79f00a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient(hunting): picked mon.noname-b con 0x7f79f0137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f0135af0 con 0x7f79f0137940 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f00a0f60 con 0x7f79f00a41b0 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79f0137730 con 0x7f79f00a49a0 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f7e4e640 10 monclient(hunting): authenticate will time out at 1218.700291s 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f53e9640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f79f00a49a0 0x7f79f01c8ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35580/0 (socket says 172.21.15.7:35580) 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79f53e9640 1 -- 172.21.15.7:0/3170989409 learned_addr learned my addr 172.21.15.7:0/3170989409 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.585+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 615446428 0 0) 0x7f79f0137730 con 0x7f79f00a49a0 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): my global_id is 95705 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79d4003520 con 0x7f79f00a49a0 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2548251371 0 0) 0x7f79f0135af0 con 0x7f79f0137940 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): my global_id is 95744 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f0137730 con 0x7f79f0137940 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2268840515 0 0) 0x7f79f00a0f60 con 0x7f79f00a41b0 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): my global_id is 76548 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f0135af0 con 0x7f79f00a41b0 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2242747226 0 0) 0x7f79d4003520 con 0x7f79f00a49a0 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79f00a0f60 con 0x7f79f00a49a0 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79e0001b10 con 0x7f79f00a49a0 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): dump: 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.606 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:30.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2060203026 0 0) 0x7f79f0137730 con 0x7f79f0137940 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79d4003520 con 0x7f79f0137940 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 771086351 0 0) 0x7f79f0135af0 con 0x7f79f00a41b0 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79f0137730 con 0x7f79f00a41b0 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79e8001af0 con 0x7f79f0137940 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): dump: 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.608 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79ec001860 con 0x7f79f00a41b0 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient(hunting): dump: 2024-05-08T15:53:30.609 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.610 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3898539131 0 0) 0x7f79f00a0f60 con 0x7f79f00a49a0 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.81:6790/0 conn(0x7f79f00a41b0 legacy=0x7f79f0137020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.7:6790/0 conn(0x7f79f0137940 legacy=0x7f79f01d47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 monclient: found mon.e 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f0133800 con 0x7f79f00a49a0 2024-05-08T15:53:30.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79f7e4e640 5 monclient: authenticate success, global_id 95705 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79f7e4e640 10 monclient: _renew_subs 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79f7e4e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f79f0135250 con 0x7f79f00a49a0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79f7e4e640 10 monclient: _renew_subs 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79f7e4e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f79f01c97c0 con 0x7f79f00a49a0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f79e00038c0 con 0x7f79f00a49a0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79e0003db0 con 0x7f79f00a49a0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e67fc640 10 monclient: dump: 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.613 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.586+0000 7f79e7fff640 4 set_mon_vals no callback set 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79f7e4e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f79d80065c0 con 0x7f79d8002860 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f79e0003f50 con 0x7f79f00a49a0 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(296..296 src has 1..296) ==== 3898+0+0 (unknown 1741224607 0 0) 0x7f79e0046ed0 con 0x7f79f00a49a0 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307410626 0 0) 0x7f79d80065c0 con 0x7f79d8002860 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 10 monclient(hunting): my global_id is 95625 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79f00a0f60 con 0x7f79d8002860 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79f63eb640 10 monclient: get_auth_request con 0x7f79d403a500 auth_method 0 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.587+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1240007340 0 0) 0x7f79f00a0f60 con 0x7f79d8002860 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79d80065c0 con 0x7f79d8002860 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79ec001cc0 con 0x7f79d8002860 2024-05-08T15:53:30.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 10 monclient: dump: 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.616 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2046883931 0 0) 0x7f79d80065c0 con 0x7f79d8002860 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f79d800a150 con 0x7f79d8002860 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f79ec005af0 con 0x7f79d8002860 2024-05-08T15:53:30.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:30.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.7:6789/0 conn(0x7f79d8002860 legacy=0x7f79d8004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.588+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.7:6789/0 conn(0x7f79d8002860 legacy=0x7f79d8004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:30.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79f7e4e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:30.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f79d8009e40 con 0x7f79d8009100 2024-05-08T15:53:30.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542790829 0 0) 0x7f79d8009e40 con 0x7f79d8009100 2024-05-08T15:53:30.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 10 monclient(hunting): my global_id is 95634 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79d80065c0 con 0x7f79d8009100 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2162872359 0 0) 0x7f79d80065c0 con 0x7f79d8009100 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f79d8009e40 con 0x7f79d8009100 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f79ec004060 con 0x7f79d8009100 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 10 monclient: dump: 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.621 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.622 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2948632499 0 0) 0x7f79d8009e40 con 0x7f79d8009100 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.595+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f79d800c3d0 con 0x7f79d8009100 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1610290457) 0x7f79ec005440 con 0x7f79d8009100 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79e67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79e67fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.7:6789/0 conn(0x7f79d8009100 legacy=0x7f79d8009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79e67fc640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.7:6789/0 conn(0x7f79d8009100 legacy=0x7f79d8009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:30.623 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 261146, 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:30.624 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 3, 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:30.625 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.626 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.627 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.628 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:30.629 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.630 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.631 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.632 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:30.635 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.636 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:30.638 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:30.639 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.640 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.641 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.644 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:30.645 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:30.646 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.648 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.649 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:30.650 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.81:6812/847627118 conn(0x7f79d403a500 legacy=0x7f79d403c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 10 monclient: shutdown 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 >> v1:172.21.15.7:6793/0 conn(0x7f79f00a49a0 legacy=0x7f79f01c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f63eb640 1 -- 172.21.15.7:0/3170989409 reap_dead start 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f63eb640 1 -- 172.21.15.7:0/3170989409 reap_dead start 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 shutdown_connections 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 >> 172.21.15.7:0/3170989409 conn(0x7f79f00a9fa0 msgr2=0x7f79f01308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 shutdown_connections 2024-05-08T15:53:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.596+0000 7f79f7e4e640 1 -- 172.21.15.7:0/3170989409 wait complete. 2024-05-08T15:53:30.653 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:30.653 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:53:30.653 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.761+0000 7febeddad640 10 monclient: get_monmap_and_config 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.761+0000 7febeddad640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.761+0000 7febeddad640 10 monclient: monmap: 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:30.762312+0000 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:30.762312+0000 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.757 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.758 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 1 Processor -- start 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 1 -- start start 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient: init 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient: _add_conns ranks=[3,2,5,4,7,1,8,6,0] 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient(hunting): picked mon.noname-g con 0x7febe8099b40 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient(hunting): picked mon.noname-b con 0x7febe809c040 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient(hunting): picked mon.noname-h con 0x7febe813e590 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe809b660 con 0x7febe809c040 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe809c420 con 0x7febe8099b40 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe809bc20 con 0x7febe813e590 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febeddad640 10 monclient(hunting): authenticate will time out at 1218.877394s 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febe6ffd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7febe809c040 0x7febe80a24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:52038/0 (socket says 172.21.15.7:52038) 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.762+0000 7febe6ffd640 1 -- 172.21.15.7:0/2821647117 learned_addr learned my addr 172.21.15.7:0/2821647117 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2595278680 0 0) 0x7febd8001110 con 0x7febe809c040 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient(hunting): my global_id is 95753 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febd4003520 con 0x7febe809c040 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1169405555 0 0) 0x7febd4003520 con 0x7febe809c040 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 >> v1:172.21.15.81:6791/0 conn(0x7febe813e590 legacy=0x7febe8140980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 >> v1:172.21.15.81:6790/0 conn(0x7febe8099b40 legacy=0x7febe80a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 monclient: found mon.noname-b 2024-05-08T15:53:30.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe80a2bf0 con 0x7febe809c040 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 5 monclient: authenticate success, global_id 95753 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7febd8001980 con 0x7febe809c040 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: dump: 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.768 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7febd8001ce0 con 0x7febe809c040 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:30.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 1 -- 172.21.15.7:0/2821647117 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7febd8004c60 con 0x7febe809c040 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 4 set_mon_vals no callback set 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febe67fc640 10 monclient: dump: 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.770 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 10 monclient: shutdown 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 1 -- 172.21.15.7:0/2821647117 >> v1:172.21.15.7:6790/0 conn(0x7febe809c040 legacy=0x7febe80a24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 1 -- 172.21.15.7:0/2821647117 shutdown_connections 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.763+0000 7febeddad640 1 -- 172.21.15.7:0/2821647117 >> 172.21.15.7:0/2821647117 conn(0x7febe80a2ec0 msgr2=0x7febe80a32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.764+0000 7febeddad640 1 -- 172.21.15.7:0/2821647117 shutdown_connections 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.764+0000 7febeddad640 1 -- 172.21.15.7:0/2821647117 wait complete. 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.764+0000 7febeddad640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.764+0000 7febeddad640 10 monclient: monmap: 2024-05-08T15:53:30.771 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.772 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.764+0000 7febeddad640 1 Processor -- start 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 1 -- start start 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient: init 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient: _add_conns ranks=[1,0,5,3,8,2,7,4,6] 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient(hunting): picked mon.noname-f con 0x7febe8099b40 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient(hunting): picked mon.noname-a con 0x7febe809c040 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient(hunting): picked mon.noname-h con 0x7febe813e590 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe809cd40 con 0x7febe809c040 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe8070350 con 0x7febe8099b40 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febe8070bb0 con 0x7febe813e590 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febeddad640 10 monclient(hunting): authenticate will time out at 1218.880344s 2024-05-08T15:53:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febe6ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7febe809c040 0x7febe81d1b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39136/0 (socket says 172.21.15.7:39136) 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febe6ffd640 1 -- 172.21.15.7:0/1347313598 learned_addr learned my addr 172.21.15.7:0/1347313598 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2245378157 0 0) 0x7febe80a2bf0 con 0x7febe809c040 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febdffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febdffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febdffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.765+0000 7febdffff640 10 monclient(hunting): my global_id is 95643 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febc8003520 con 0x7febe809c040 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2042193204 0 0) 0x7febe8070350 con 0x7febe8099b40 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): my global_id is 6184 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe80a2bf0 con 0x7febe8099b40 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2168730119 0 0) 0x7febe8070bb0 con 0x7febe813e590 2024-05-08T15:53:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): my global_id is 6395 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe8070350 con 0x7febe813e590 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3068495197 0 0) 0x7febc8003520 con 0x7febe809c040 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7febe8070bb0 con 0x7febe809c040 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7febd8004b20 con 0x7febe809c040 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): dump: 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.775 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.776 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1613907018 0 0) 0x7febe80a2bf0 con 0x7febe8099b40 2024-05-08T15:53:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7febc8003520 con 0x7febe8099b40 2024-05-08T15:53:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7febd0001ba0 con 0x7febe8099b40 2024-05-08T15:53:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient(hunting): dump: 2024-05-08T15:53:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.780 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2225586949 0 0) 0x7febe8070bb0 con 0x7febe809c040 2024-05-08T15:53:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.81:6791/0 conn(0x7febe813e590 legacy=0x7febe81d5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.81:6789/0 conn(0x7febe8099b40 legacy=0x7febe81cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 monclient: found mon.a 2024-05-08T15:53:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:53:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe8071870 con 0x7febe809c040 2024-05-08T15:53:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febeddad640 5 monclient: authenticate success, global_id 95643 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febeddad640 10 monclient: _renew_subs 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febeddad640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7febe806f5b0 con 0x7febe809c040 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febeddad640 10 monclient: _renew_subs 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febeddad640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:53:30.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7febe813d390 con 0x7febe809c040 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7febd8001ce0 con 0x7febe809c040 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7febd80051d0 con 0x7febe809c040 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febe5ffb640 4 set_mon_vals no callback set 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.766+0000 7febdffff640 10 monclient: dump: 2024-05-08T15:53:30.783 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.785 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.785 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febeddad640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7febcc0065c0 con 0x7febcc002860 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7febd8010550 con 0x7febe809c040 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(296..296 src has 1..296) ==== 3898+0+0 (unknown 1741224607 0 0) 0x7febd8046f10 con 0x7febe809c040 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676128603 0 0) 0x7febcc0065c0 con 0x7febcc002860 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 10 monclient(hunting): my global_id is 95762 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febe8070bb0 con 0x7febcc002860 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.767+0000 7febe7fff640 10 monclient: get_auth_request con 0x7febc8039fd0 auth_method 0 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 427842617 0 0) 0x7febe8070bb0 con 0x7febcc002860 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7febcc0065c0 con 0x7febcc002860 2024-05-08T15:53:30.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7febd00011b0 con 0x7febcc002860 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 10 monclient: dump: 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.787 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2451642010 0 0) 0x7febcc0065c0 con 0x7febcc002860 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febcc00a150 con 0x7febcc002860 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7febd0003f90 con 0x7febcc002860 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.7:6790/0 conn(0x7febcc002860 legacy=0x7febcc004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.768+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.7:6790/0 conn(0x7febcc002860 legacy=0x7febcc004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febeddad640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7febcc009e20 con 0x7febcc0090e0 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1313680940 0 0) 0x7febcc009e20 con 0x7febcc0090e0 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient(hunting): my global_id is 95771 2024-05-08T15:53:30.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7febcc0065c0 con 0x7febcc0090e0 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 327920785 0 0) 0x7febcc0065c0 con 0x7febcc0090e0 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7febcc009e20 con 0x7febcc0090e0 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7febd0004810 con 0x7febcc0090e0 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient: dump: 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.791 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 487310666 0 0) 0x7febcc009e20 con 0x7febcc0090e0 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7febcc00c390 con 0x7febcc0090e0 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 3916206555) 0x7febd0005b40 con 0x7febcc0090e0 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:30.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.7:6790/0 conn(0x7febcc0090e0 legacy=0x7febcc009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.775+0000 7febdffff640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.7:6790/0 conn(0x7febcc0090e0 legacy=0x7febcc009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 261326, 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:30.793 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:30.794 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:30.795 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:30.797 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.798 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.799 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:30.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.802 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:30.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:30.808 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.809 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.810 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.811 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:53:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.812 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:30.812 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:30.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.81:6812/847627118 conn(0x7febc8039fd0 legacy=0x7febc803c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 10 monclient: shutdown 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 >> v1:172.21.15.7:6789/0 conn(0x7febe809c040 legacy=0x7febe81d1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febe7fff640 1 -- 172.21.15.7:0/1347313598 reap_dead start 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febe7fff640 1 -- 172.21.15.7:0/1347313598 reap_dead start 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 shutdown_connections 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 >> 172.21.15.7:0/1347313598 conn(0x7febe80a2ec0 msgr2=0x7febe813ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 shutdown_connections 2024-05-08T15:53:30.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.776+0000 7febeddad640 1 -- 172.21.15.7:0/1347313598 wait complete. 2024-05-08T15:53:30.814 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:30.815 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:53:30.815 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38907]: starting. 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38907]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38868 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38907]: created object 0... 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38907]: created object 25... 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38907]: created object 49... 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38907]: finishing. 2024-05-08T15:53:30.875 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[38907]: shutting down. 2024-05-08T15:53:30.885 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:30.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38909]: starting. 2024-05-08T15:53:30.885 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38909]: listing objects. 2024-05-08T15:53:30.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38909]: listed object 0... 2024-05-08T15:53:30.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38909]: listed object 25... 2024-05-08T15:53:30.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38909]: saw 50 objects 2024-05-08T15:53:30.886 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[38909]: shutting down. 2024-05-08T15:53:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.931+0000 7f945f9bd640 10 monclient: get_monmap_and_config 2024-05-08T15:53:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.931+0000 7f945f9bd640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.931+0000 7f945f9bd640 10 monclient: monmap: 2024-05-08T15:53:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:30.932893+0000 2024-05-08T15:53:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:30.932893+0000 2024-05-08T15:53:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.930 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.932+0000 7f945f9bd640 1 Processor -- start 2024-05-08T15:53:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.932+0000 7f945f9bd640 1 -- start start 2024-05-08T15:53:30.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.932+0000 7f945f9bd640 10 monclient: init 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 10 monclient: _add_conns ranks=[6,2,4,8,1,3,5,0,7] 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 10 monclient(hunting): picked mon.noname-d con 0x7f94580a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 10 monclient(hunting): picked mon.noname-b con 0x7f94580a5780 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 10 monclient(hunting): picked mon.noname-c con 0x7f945813e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94580a1660 con 0x7f94580a5780 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94580a28f0 con 0x7f945813e310 2024-05-08T15:53:30.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94580a2bd0 con 0x7f94580a53a0 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945f9bd640 10 monclient(hunting): authenticate will time out at 1219.048069s 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945cf58640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f94580a5780 0x7f945813cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:52076/0 (socket says 172.21.15.7:52076) 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945cf58640 1 -- 172.21.15.7:0/567685799 learned_addr learned my addr 172.21.15.7:0/567685799 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f945d759640 1 --1- 172.21.15.7:0/567685799 >> v1:172.21.15.7:6792/0 conn(0x7f94580a53a0 0x7f94580a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35364/0 (socket says 172.21.15.7:35364) 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1067407650 0 0) 0x7f94580a28f0 con 0x7f945813e310 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): my global_id is 124195 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9444003520 con 0x7f945813e310 2024-05-08T15:53:30.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4247429725 0 0) 0x7f94580a2bd0 con 0x7f94580a53a0 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): my global_id is 124233 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94580a28f0 con 0x7f94580a53a0 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 628135333 0 0) 0x7f94580a1660 con 0x7f94580a5780 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 10 monclient(hunting): my global_id is 95780 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.933+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94580a2bd0 con 0x7f94580a5780 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2066026659 0 0) 0x7f9444003520 con 0x7f945813e310 2024-05-08T15:53:30.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 >> v1:172.21.15.7:6792/0 conn(0x7f94580a53a0 legacy=0x7f94580a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 >> v1:172.21.15.7:6790/0 conn(0x7f94580a5780 legacy=0x7f945813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 monclient: found mon.noname-c 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94580a6730 con 0x7f945813e310 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9450001b30 con 0x7f945813e310 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: dump: 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.947 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 5 monclient: authenticate success, global_id 124195 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9450001d90 con 0x7f945813e310 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 1 -- 172.21.15.7:0/567685799 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9450003cd0 con 0x7f945813e310 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 4 set_mon_vals no callback set 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f944ffff640 10 monclient: dump: 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.949 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 10 monclient: shutdown 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 1 -- 172.21.15.7:0/567685799 >> v1:172.21.15.7:6791/0 conn(0x7f945813e310 legacy=0x7f9458140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 1 -- 172.21.15.7:0/567685799 shutdown_connections 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 1 -- 172.21.15.7:0/567685799 >> 172.21.15.7:0/567685799 conn(0x7f94580a9fa0 msgr2=0x7f94580aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 1 -- 172.21.15.7:0/567685799 shutdown_connections 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.934+0000 7f945f9bd640 1 -- 172.21.15.7:0/567685799 wait complete. 2024-05-08T15:53:30.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.935+0000 7f945f9bd640 10 monclient: build_initial_monmap 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.935+0000 7f945f9bd640 10 monclient: monmap: 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:30.951 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.935+0000 7f945f9bd640 1 Processor -- start 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.935+0000 7f945f9bd640 1 -- start start 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.935+0000 7f945f9bd640 10 monclient: init 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 10 monclient: _add_conns ranks=[7,4,3,6,8,1,2,5,0] 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 10 monclient(hunting): picked mon.noname-i con 0x7f94580a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 10 monclient(hunting): picked mon.noname-c con 0x7f94580a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:30.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 10 monclient(hunting): picked mon.noname-g con 0x7f945813e310 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94581377f0 con 0x7f945813e310 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94581ccf80 con 0x7f94580a5780 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94580a43c0 con 0x7f94580a53a0 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945f9bd640 10 monclient(hunting): authenticate will time out at 1219.050944s 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945cf58640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f94580a5780 0x7f94581cc870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51906/0 (socket says 172.21.15.7:51906) 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f945cf58640 1 -- 172.21.15.7:0/3275573865 learned_addr learned my addr 172.21.15.7:0/3275573865 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1597777421 0 0) 0x7f94581ccf80 con 0x7f94580a5780 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): my global_id is 124204 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f943c003520 con 0x7f94580a5780 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838596441 0 0) 0x7f94581377f0 con 0x7f945813e310 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): my global_id is 76566 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94581ccf80 con 0x7f945813e310 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3933704863 0 0) 0x7f94580a43c0 con 0x7f94580a53a0 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): my global_id is 114928 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94581377f0 con 0x7f94580a53a0 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3365186302 0 0) 0x7f943c003520 con 0x7f94580a5780 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f94580a43c0 con 0x7f94580a5780 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9448001670 con 0x7f94580a5780 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): dump: 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.955 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.936+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4079286585 0 0) 0x7f94581ccf80 con 0x7f945813e310 2024-05-08T15:53:30.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f943c003520 con 0x7f945813e310 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9450001a30 con 0x7f945813e310 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient(hunting): dump: 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.957 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1453596985 0 0) 0x7f94581377f0 con 0x7f94580a53a0 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f94581ccf80 con 0x7f94580a53a0 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 662210100 0 0) 0x7f94580a43c0 con 0x7f94580a5780 2024-05-08T15:53:30.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.81:6792/0 conn(0x7f94580a53a0 legacy=0x7f94581cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.81:6790/0 conn(0x7f945813e310 legacy=0x7f94581d46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: _finish_hunting 0 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 monclient: found mon.c 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9458137510 con 0x7f94580a5780 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: _finish_auth 0 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: _check_auth_tickets 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f945f9bd640 5 monclient: authenticate success, global_id 124204 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f945f9bd640 10 monclient: _renew_subs 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f945f9bd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9458136060 con 0x7f94580a5780 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9448004a90 con 0x7f94580a5780 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9448004f80 con 0x7f94580a5780 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944dffb640 10 monclient: dump: 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.960 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f944f7fe640 4 set_mon_vals no callback set 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f945f9bd640 10 monclient: _renew_subs 2024-05-08T15:53:30.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f945f9bd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.937+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f9458133830 con 0x7f94580a5780 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f945f9bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f9448010300 con 0x7f94580a5780 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9458075820 con 0x7f94581399d0 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(297..297 src has 1..297) ==== 3898+0+0 (unknown 2395490465 0 0) 0x7f9448046ef0 con 0x7f94580a5780 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515348596 0 0) 0x7f9458075820 con 0x7f94581399d0 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 10 monclient(hunting): my global_id is 124213 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94580a43c0 con 0x7f94581399d0 2024-05-08T15:53:30.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2049933914 0 0) 0x7f94580a43c0 con 0x7f94581399d0 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9458075820 con 0x7f94581399d0 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9454001d20 con 0x7f94581399d0 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f945df5a640 10 monclient: get_auth_request con 0x7f943c03a3a0 auth_method 0 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.938+0000 7f944dffb640 10 monclient: dump: 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.963 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.939+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3839549804 0 0) 0x7f9458075820 con 0x7f94581399d0 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.939+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94580a1e00 con 0x7f94581399d0 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.939+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9454004e20 con 0x7f94581399d0 2024-05-08T15:53:30.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.939+0000 7f944dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.939+0000 7f944dffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.939+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.7:6791/0 conn(0x7f94581399d0 legacy=0x7f9458139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.939+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.7:6791/0 conn(0x7f94581399d0 legacy=0x7f9458139db0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.945+0000 7f945f9bd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.945+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f945806b380 con 0x7f945813a4c0 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4014334560 0 0) 0x7f945806b380 con 0x7f945813a4c0 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient(hunting): my global_id is 124222 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9458075820 con 0x7f945813a4c0 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2195882536 0 0) 0x7f9458075820 con 0x7f945813a4c0 2024-05-08T15:53:30.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f945806b380 con 0x7f945813a4c0 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9454005540 con 0x7f945813a4c0 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient: dump: 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:30.966 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 543318715 0 0) 0x7f945806b380 con 0x7f945813a4c0 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f945813a8a0 con 0x7f945813a4c0 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7522 (unknown 0 0 1468880904) 0x7f9454006990 con 0x7f945813a4c0 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:30.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.7:6791/0 conn(0x7f945813a4c0 legacy=0x7f94581d9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.946+0000 7f944dffb640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.7:6791/0 conn(0x7f945813a4c0 legacy=0x7f94581d9df0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:30.968 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 19116, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.969 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.970 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.971 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:30.972 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:30.973 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:30.974 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:30.975 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:30.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:30.977 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:30.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.981 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:30.986 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.987 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:30.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:30.989 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:30.989 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:30.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.81:6812/847627118 conn(0x7f943c03a3a0 legacy=0x7f943c03c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 10 monclient: shutdown 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 >> v1:172.21.15.7:6791/0 conn(0x7f94580a5780 legacy=0x7f94581cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945df5a640 1 -- 172.21.15.7:0/3275573865 reap_dead start 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945df5a640 1 -- 172.21.15.7:0/3275573865 reap_dead start 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 shutdown_connections 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 >> 172.21.15.7:0/3275573865 conn(0x7f94580a9fa0 msgr2=0x7f94580a8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 shutdown_connections 2024-05-08T15:53:30.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:30.947+0000 7f945f9bd640 1 -- 172.21.15.7:0/3275573865 wait complete. 2024-05-08T15:53:30.992 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:30.992 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:53:30.992 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:53:31.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.100+0000 7f1064f60640 10 monclient: get_monmap_and_config 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.100+0000 7f1064f60640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.100+0000 7f1064f60640 10 monclient: monmap: 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:31.101115+0000 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:31.101115+0000 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.096 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.097 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.100+0000 7f1064f60640 1 Processor -- start 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.100+0000 7f1064f60640 1 -- start start 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.100+0000 7f1064f60640 10 monclient: init 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 10 monclient: _add_conns ranks=[3,4,7,5,6,0,2,8,1] 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 10 monclient(hunting): picked mon.noname-g con 0x7f10600a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 10 monclient(hunting): picked mon.noname-c con 0x7f10600a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 10 monclient(hunting): picked mon.noname-i con 0x7f106013e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600a1660 con 0x7f10600a53a0 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600a28f0 con 0x7f10600a5780 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10600a2bd0 con 0x7f106013e310 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f1064f60640 10 monclient(hunting): authenticate will time out at 1219.216077s 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105dd9b640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f10600a5780 0x7f106013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51932/0 (socket says 172.21.15.7:51932) 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105dd9b640 1 -- 172.21.15.7:0/3628396727 learned_addr learned my addr 172.21.15.7:0/3628396727 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2569716629 0 0) 0x7f10600a28f0 con 0x7f10600a5780 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): my global_id is 124231 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1048003520 con 0x7f10600a5780 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1609854875 0 0) 0x7f10600a1660 con 0x7f10600a53a0 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): my global_id is 76575 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10600a28f0 con 0x7f10600a53a0 2024-05-08T15:53:31.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573137004 0 0) 0x7f10600a2bd0 con 0x7f106013e310 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 10 monclient(hunting): my global_id is 114937 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.101+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10600a1660 con 0x7f106013e310 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1339942912 0 0) 0x7f1048003520 con 0x7f10600a5780 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 >> v1:172.21.15.81:6792/0 conn(0x7f106013e310 legacy=0x7f1060140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 >> v1:172.21.15.81:6790/0 conn(0x7f10600a53a0 legacy=0x7f10600a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 monclient: found mon.noname-c 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:53:31.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10600a6730 con 0x7f10600a5780 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1054001b30 con 0x7f10600a5780 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 5 monclient: authenticate success, global_id 124231 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: dump: 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.110 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1054001e50 con 0x7f10600a5780 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 1 -- 172.21.15.7:0/3628396727 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1054004f60 con 0x7f10600a5780 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 4 set_mon_vals no callback set 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f105d59a640 10 monclient: dump: 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.112 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 10 monclient: shutdown 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 1 -- 172.21.15.7:0/3628396727 >> v1:172.21.15.7:6791/0 conn(0x7f10600a5780 legacy=0x7f106013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 1 -- 172.21.15.7:0/3628396727 shutdown_connections 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 1 -- 172.21.15.7:0/3628396727 >> 172.21.15.7:0/3628396727 conn(0x7f10600a9fa0 msgr2=0x7f10600aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 1 -- 172.21.15.7:0/3628396727 shutdown_connections 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.102+0000 7f1064f60640 1 -- 172.21.15.7:0/3628396727 wait complete. 2024-05-08T15:53:31.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.103+0000 7f1064f60640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.103+0000 7f1064f60640 10 monclient: monmap: 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.114 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.103+0000 7f1064f60640 1 Processor -- start 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.103+0000 7f1064f60640 1 -- start start 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.103+0000 7f1064f60640 10 monclient: init 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 10 monclient: _add_conns ranks=[4,5,6,1,3,2,0,7,8] 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 10 monclient(hunting): picked mon.noname-c con 0x7f10600a53a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 10 monclient(hunting): picked mon.noname-h con 0x7f10600a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 10 monclient(hunting): picked mon.noname-d con 0x7f106013e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10601351a0 con 0x7f10600a53a0 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10601347c0 con 0x7f10600a5780 2024-05-08T15:53:31.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f105e59c640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f10600a53a0 0x7f1060132fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51942/0 (socket says 172.21.15.7:51942) 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f105e59c640 1 -- 172.21.15.7:0/4104692608 learned_addr learned my addr 172.21.15.7:0/4104692608 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1060134d80 con 0x7f106013e310 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f1064f60640 10 monclient(hunting): authenticate will time out at 1219.219055s 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549072631 0 0) 0x7f10600a28f0 con 0x7f10600a53a0 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): my global_id is 124240 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1040003520 con 0x7f10600a53a0 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638433518 0 0) 0x7f1060134d80 con 0x7f106013e310 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): my global_id is 124242 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10600a28f0 con 0x7f106013e310 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2093761793 0 0) 0x7f10601347c0 con 0x7f10600a5780 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): my global_id is 6404 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1060134d80 con 0x7f10600a5780 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1270671944 0 0) 0x7f1040003520 con 0x7f10600a53a0 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f10601347c0 con 0x7f10600a53a0 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1050001a50 con 0x7f10600a53a0 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): dump: 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.118 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3901960796 0 0) 0x7f10600a28f0 con 0x7f106013e310 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.104+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1040003520 con 0x7f106013e310 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f10580019b0 con 0x7f106013e310 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient(hunting): dump: 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.119 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1300673450 0 0) 0x7f1060134d80 con 0x7f10600a5780 2024-05-08T15:53:31.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f10600a28f0 con 0x7f10600a5780 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1054004ea0 con 0x7f10600a5780 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient(hunting): dump: 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.121 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3870685810 0 0) 0x7f10601347c0 con 0x7f10600a53a0 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.7:6792/0 conn(0x7f106013e310 legacy=0x7f10601d4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.81:6791/0 conn(0x7f10600a5780 legacy=0x7f10601d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 monclient: found mon.c 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:31.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1060134aa0 con 0x7f10600a53a0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f1064f60640 5 monclient: authenticate success, global_id 124240 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f1064f60640 10 monclient: _renew_subs 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f1064f60640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f10601336d0 con 0x7f10600a53a0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f1064f60640 10 monclient: _renew_subs 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f1064f60640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f10601d5d60 con 0x7f10600a53a0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f10500038c0 con 0x7f10600a53a0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1050003e30 con 0x7f10600a53a0 2024-05-08T15:53:31.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 10 monclient: dump: 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.124 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f105cd99640 4 set_mon_vals no callback set 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.105+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f1050004050 con 0x7f10600a53a0 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(297..297 src has 1..297) ==== 3898+0+0 (unknown 2395490465 0 0) 0x7f1050046ea0 con 0x7f10600a53a0 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f1064f60640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f10601338a0 con 0x7f10601400f0 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f105dd9b640 10 monclient: get_auth_request con 0x7f104003a3d0 auth_method 0 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3622560192 0 0) 0x7f10601338a0 con 0x7f10601400f0 2024-05-08T15:53:31.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 10 monclient(hunting): my global_id is 124251 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10601347c0 con 0x7f10601400f0 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2510274768 0 0) 0x7f10601347c0 con 0x7f10601400f0 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f10601338a0 con 0x7f10601400f0 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1058003950 con 0x7f10601400f0 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 10 monclient: dump: 2024-05-08T15:53:31.126 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.127 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 470891661 0 0) 0x7f10601338a0 con 0x7f10601400f0 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.106+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1060132d50 con 0x7f10601400f0 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.107+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f1058004f50 con 0x7f10601400f0 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.107+0000 7f104effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.107+0000 7f104effd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.107+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.7:6792/0 conn(0x7f10601400f0 legacy=0x7f10601404d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.107+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.7:6792/0 conn(0x7f10601400f0 legacy=0x7f10601404d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.113+0000 7f1064f60640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.113+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f1060075820 con 0x7f106002d050 2024-05-08T15:53:31.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2613806877 0 0) 0x7f1060075820 con 0x7f106002d050 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient(hunting): my global_id is 124260 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f10601338a0 con 0x7f106002d050 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3155953190 0 0) 0x7f10601338a0 con 0x7f106002d050 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1060075820 con 0x7f106002d050 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1058005750 con 0x7f106002d050 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient: dump: 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.130 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2528873229 0 0) 0x7f1060075820 con 0x7f106002d050 2024-05-08T15:53:31.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f106006b380 con 0x7f106002d050 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7522 (unknown 0 0 3204777776) 0x7f1058003950 con 0x7f106002d050 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.7:6792/0 conn(0x7f106002d050 legacy=0x7f1060140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.114+0000 7f104effd640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.7:6792/0 conn(0x7f106002d050 legacy=0x7f1060140be0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:31.132 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 19284, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:31.133 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:31.134 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:31.135 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.136 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:31.137 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.138 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:31.139 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.140 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.141 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.142 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.143 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.144 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:31.145 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.146 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.147 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.148 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.149 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:31.150 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.81:6812/847627118 conn(0x7f104003a3d0 legacy=0x7f104003c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 10 monclient: shutdown 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 >> v1:172.21.15.7:6791/0 conn(0x7f10600a53a0 legacy=0x7f1060132fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f105ed9d640 1 -- 172.21.15.7:0/4104692608 reap_dead start 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f105ed9d640 1 -- 172.21.15.7:0/4104692608 reap_dead start 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 shutdown_connections 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 >> 172.21.15.7:0/4104692608 conn(0x7f10600a9fa0 msgr2=0x7f10600a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 shutdown_connections 2024-05-08T15:53:31.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.115+0000 7f1064f60640 1 -- 172.21.15.7:0/4104692608 wait complete. 2024-05-08T15:53:31.153 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:31.153 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:53:31.153 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.261+0000 7fce36979640 10 monclient: get_monmap_and_config 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.261+0000 7fce36979640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.261+0000 7fce36979640 10 monclient: monmap: 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:31.262313+0000 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:31.262313+0000 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.257 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.258 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 1 Processor -- start 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 1 -- start start 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient: init 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient: _add_conns ranks=[0,4,1,2,5,3,8,7,6] 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient(hunting): picked mon.noname-a con 0x7fce300a5430 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient(hunting): picked mon.noname-c con 0x7fce300a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient(hunting): picked mon.noname-f con 0x7fce3013e380 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:31.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300a2990 con 0x7fce300a5430 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300a2c70 con 0x7fce3013e380 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300a2f50 con 0x7fce300a49a0 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.262+0000 7fce36979640 10 monclient(hunting): authenticate will time out at 1219.377537s 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2ffff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fce300a5430 0x7fce300a5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:39142/0 (socket says 172.21.15.7:39142) 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2ffff640 1 -- 172.21.15.7:0/2660526006 learned_addr learned my addr 172.21.15.7:0/2660526006 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce27fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fce300a49a0 0x7fce3013cc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51944/0 (socket says 172.21.15.7:51944) 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 338226629 0 0) 0x7fce300a2990 con 0x7fce300a5430 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): my global_id is 95652 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce1c003520 con 0x7fce300a5430 2024-05-08T15:53:31.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4264800898 0 0) 0x7fce300a2f50 con 0x7fce300a49a0 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): my global_id is 124249 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce300a2990 con 0x7fce300a49a0 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1137255564 0 0) 0x7fce300a2c70 con 0x7fce3013e380 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient(hunting): my global_id is 6193 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce300a2f50 con 0x7fce3013e380 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2743301996 0 0) 0x7fce1c003520 con 0x7fce300a5430 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 >> v1:172.21.15.7:6791/0 conn(0x7fce300a49a0 legacy=0x7fce3013cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 >> v1:172.21.15.81:6789/0 conn(0x7fce3013e380 legacy=0x7fce30140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 monclient: found mon.noname-a 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce300a4300 con 0x7fce300a5430 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce36979640 5 monclient: authenticate success, global_id 95652 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce36979640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce200017b0 con 0x7fce300a5430 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: dump: 2024-05-08T15:53:31.270 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.271 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce36979640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fce20001a10 con 0x7fce300a5430 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce36979640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce36979640 4 set_mon_vals no callback set 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 1 -- 172.21.15.7:0/2660526006 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce20004be0 con 0x7fce300a5430 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.263+0000 7fce2f7fe640 10 monclient: dump: 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.272 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 10 monclient: shutdown 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 1 -- 172.21.15.7:0/2660526006 >> v1:172.21.15.7:6789/0 conn(0x7fce300a5430 legacy=0x7fce300a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 1 -- 172.21.15.7:0/2660526006 shutdown_connections 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 1 -- 172.21.15.7:0/2660526006 >> 172.21.15.7:0/2660526006 conn(0x7fce300a9fa0 msgr2=0x7fce300aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 1 -- 172.21.15.7:0/2660526006 shutdown_connections 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 1 -- 172.21.15.7:0/2660526006 wait complete. 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.264+0000 7fce36979640 10 monclient: monmap: 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.274 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 1 Processor -- start 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 1 -- start start 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient: init 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient: _add_conns ranks=[8,4,1,2,0,5,6,3,7] 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient(hunting): picked mon.noname-e con 0x7fce300a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:31.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient(hunting): picked mon.noname-c con 0x7fce300a5430 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient(hunting): picked mon.noname-f con 0x7fce3013e380 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300a1f30 con 0x7fce3013e380 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce3009e6a0 con 0x7fce300a5430 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce300a0f60 con 0x7fce300a49a0 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce36979640 10 monclient(hunting): authenticate will time out at 1219.380321s 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce2ffff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fce300a49a0 0x7fce301cc4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35586/0 (socket says 172.21.15.7:35586) 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.265+0000 7fce2ffff640 1 -- 172.21.15.7:0/989311922 learned_addr learned my addr 172.21.15.7:0/989311922 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106240709 0 0) 0x7fce3009e6a0 con 0x7fce300a5430 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): my global_id is 124258 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce10003520 con 0x7fce300a5430 2024-05-08T15:53:31.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820745081 0 0) 0x7fce300a0f60 con 0x7fce300a49a0 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): my global_id is 95714 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce3009e6a0 con 0x7fce300a49a0 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1765274792 0 0) 0x7fce300a2f50 con 0x7fce3013e380 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): my global_id is 6202 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce300a0f60 con 0x7fce3013e380 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2372845510 0 0) 0x7fce10003520 con 0x7fce300a5430 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fce300a2f50 con 0x7fce300a5430 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce18001450 con 0x7fce300a5430 2024-05-08T15:53:31.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): dump: 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.278 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.279 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3160454910 0 0) 0x7fce300a0f60 con 0x7fce3013e380 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fce10003520 con 0x7fce3013e380 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1725667891 0 0) 0x7fce3009e6a0 con 0x7fce300a49a0 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fce300a0f60 con 0x7fce300a49a0 2024-05-08T15:53:31.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce28001cb0 con 0x7fce3013e380 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): dump: 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.281 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce20004b20 con 0x7fce300a49a0 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient(hunting): dump: 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.283 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1072025947 0 0) 0x7fce300a2f50 con 0x7fce300a5430 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.7:6793/0 conn(0x7fce300a49a0 legacy=0x7fce301cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.81:6789/0 conn(0x7fce3013e380 legacy=0x7fce301d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 monclient: found mon.c 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce301ccbe0 con 0x7fce300a5430 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce36979640 5 monclient: authenticate success, global_id 124258 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce36979640 10 monclient: _renew_subs 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce36979640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fce3009d1f0 con 0x7fce300a5430 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce36979640 10 monclient: _renew_subs 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce36979640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:31.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fce3009e3c0 con 0x7fce300a5430 2024-05-08T15:53:31.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fce180016d0 con 0x7fce300a5430 2024-05-08T15:53:31.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce18004dd0 con 0x7fce300a5430 2024-05-08T15:53:31.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2d7fa640 10 monclient: dump: 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.287 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.266+0000 7fce2effd640 4 set_mon_vals no callback set 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fce18010150 con 0x7fce300a5430 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce36979640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fce140065c0 con 0x7fce14002860 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(297..297 src has 1..297) ==== 3898+0+0 (unknown 2395490465 0 0) 0x7fce18046d60 con 0x7fce300a5430 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1949397929 0 0) 0x7fce140065c0 con 0x7fce14002860 2024-05-08T15:53:31.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 10 monclient(hunting): my global_id is 95723 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce300a2f50 con 0x7fce14002860 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce34f16640 10 monclient: get_auth_request con 0x7fce1003a410 auth_method 0 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.267+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 607963809 0 0) 0x7fce300a2f50 con 0x7fce14002860 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fce140065c0 con 0x7fce14002860 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce20001d00 con 0x7fce14002860 2024-05-08T15:53:31.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 10 monclient: dump: 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.290 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1811278467 0 0) 0x7fce140065c0 con 0x7fce14002860 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce1400a150 con 0x7fce14002860 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fce20005230 con 0x7fce14002860 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:31.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.7:6793/0 conn(0x7fce14002860 legacy=0x7fce14004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.268+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.7:6793/0 conn(0x7fce14002860 legacy=0x7fce14004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.274+0000 7fce36979640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:31.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fce14009e40 con 0x7fce14009100 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3366417913 0 0) 0x7fce14009e40 con 0x7fce14009100 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient(hunting): my global_id is 95732 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce140065c0 con 0x7fce14009100 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 723639324 0 0) 0x7fce140065c0 con 0x7fce14009100 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fce14009e40 con 0x7fce14009100 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fce20005990 con 0x7fce14009100 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:31.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient: dump: 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.295 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2652569162 0 0) 0x7fce14009e40 con 0x7fce14009100 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fce1400c3d0 con 0x7fce14009100 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 176087835) 0x7fce200086e0 con 0x7fce14009100 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.7:6793/0 conn(0x7fce14009100 legacy=0x7fce14009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.275+0000 7fce2d7fa640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.7:6793/0 conn(0x7fce14009100 legacy=0x7fce14009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.297 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 261827, 2024-05-08T15:53:31.297 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:31.298 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.299 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.300 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.301 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.302 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.303 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.304 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.305 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.306 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.307 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.308 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.309 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.310 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.311 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.312 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.313 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:31.314 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.315 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:53:31.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:31.317 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:31.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.81:6812/847627118 conn(0x7fce1003a410 legacy=0x7fce1003c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 10 monclient: shutdown 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 >> v1:172.21.15.7:6791/0 conn(0x7fce300a5430 legacy=0x7fce3009feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce34f16640 1 -- 172.21.15.7:0/989311922 reap_dead start 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce34f16640 1 -- 172.21.15.7:0/989311922 reap_dead start 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 shutdown_connections 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 >> 172.21.15.7:0/989311922 conn(0x7fce300a9fa0 msgr2=0x7fce30136ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 shutdown_connections 2024-05-08T15:53:31.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.276+0000 7fce36979640 1 -- 172.21.15.7:0/989311922 wait complete. 2024-05-08T15:53:31.319 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:31.320 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:53:31.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.427+0000 7fd18f40f640 10 monclient: get_monmap_and_config 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.427+0000 7fd18f40f640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.427+0000 7fd18f40f640 10 monclient: monmap: 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:31.428849+0000 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:31.428849+0000 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.424 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.428+0000 7fd18f40f640 1 Processor -- start 2024-05-08T15:53:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.428+0000 7fd18f40f640 1 -- start start 2024-05-08T15:53:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.428+0000 7fd18f40f640 10 monclient: init 2024-05-08T15:53:31.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 10 monclient: _add_conns ranks=[6,2,1,7,5,3,8,0,4] 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 10 monclient(hunting): picked mon.noname-d con 0x7fd188137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 10 monclient(hunting): picked mon.noname-b con 0x7fd1880a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 10 monclient(hunting): picked mon.noname-f con 0x7fd1880a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1880a2c70 con 0x7fd1880a41b0 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1880a2f50 con 0x7fd1880a49a0 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1880a1670 con 0x7fd188137940 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 10 monclient(hunting): authenticate will time out at 1219.543796s 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18d1ab640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fd188137940 0x7fd188137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35388/0 (socket says 172.21.15.7:35388) 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18d1ab640 1 -- 172.21.15.7:0/428996849 learned_addr learned my addr 172.21.15.7:0/428996849 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18c9aa640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd1880a49a0 0x7fd18813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:52092/0 (socket says 172.21.15.7:52092) 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3288492819 0 0) 0x7fd1880a2f50 con 0x7fd1880a49a0 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): my global_id is 95789 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd174003520 con 0x7fd1880a49a0 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3460820823 0 0) 0x7fd1880a1670 con 0x7fd188137940 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): my global_id is 124269 2024-05-08T15:53:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1880a2f50 con 0x7fd188137940 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258953227 0 0) 0x7fd1880a2c70 con 0x7fd1880a41b0 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient(hunting): my global_id is 6211 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1880a1670 con 0x7fd1880a41b0 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1740641664 0 0) 0x7fd174003520 con 0x7fd1880a49a0 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 >> v1:172.21.15.7:6792/0 conn(0x7fd188137940 legacy=0x7fd188137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 >> v1:172.21.15.81:6789/0 conn(0x7fd1880a41b0 legacy=0x7fd188140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 monclient: found mon.noname-b 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1880a39f0 con 0x7fd1880a49a0 2024-05-08T15:53:31.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd1780017b0 con 0x7fd1880a49a0 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 5 monclient: authenticate success, global_id 95789 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd17ffff640 10 monclient: dump: 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.437 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.429+0000 7fd18f40f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd178001ad0 con 0x7fd1880a49a0 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd17ffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd17ffff640 1 -- 172.21.15.7:0/428996849 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd178001d40 con 0x7fd1880a49a0 2024-05-08T15:53:31.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 4 set_mon_vals no callback set 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd17ffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd17ffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd17ffff640 10 monclient: dump: 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.439 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 10 monclient: shutdown 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 1 -- 172.21.15.7:0/428996849 >> v1:172.21.15.7:6790/0 conn(0x7fd1880a49a0 legacy=0x7fd18813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 1 -- 172.21.15.7:0/428996849 shutdown_connections 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 1 -- 172.21.15.7:0/428996849 >> 172.21.15.7:0/428996849 conn(0x7fd1880a9fa0 msgr2=0x7fd1880aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 1 -- 172.21.15.7:0/428996849 shutdown_connections 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.430+0000 7fd18f40f640 1 -- 172.21.15.7:0/428996849 wait complete. 2024-05-08T15:53:31.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 10 monclient: monmap: 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.441 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 1 Processor -- start 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 1 -- start start 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 10 monclient: init 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 10 monclient: _add_conns ranks=[5,2,0,8,3,7,1,4,6] 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 10 monclient(hunting): picked mon.noname-h con 0x7fd1880a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.431+0000 7fd18f40f640 10 monclient(hunting): picked mon.noname-b con 0x7fd1880a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 10 monclient(hunting): picked mon.noname-a con 0x7fd188137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1880a0f60 con 0x7fd188137940 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd188134180 con 0x7fd1880a49a0 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd188135070 con 0x7fd1880a41b0 2024-05-08T15:53:31.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 10 monclient(hunting): authenticate will time out at 1219.546641s 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18c9aa640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd1880a49a0 0x7fd188136e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:52104/0 (socket says 172.21.15.7:52104) 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18c9aa640 1 -- 172.21.15.7:0/4088061734 learned_addr learned my addr 172.21.15.7:0/4088061734 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1676766878 0 0) 0x7fd188134180 con 0x7fd1880a49a0 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): my global_id is 95798 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd16c003550 con 0x7fd1880a49a0 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832378222 0 0) 0x7fd1880a0f60 con 0x7fd188137940 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): my global_id is 95661 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd188134180 con 0x7fd188137940 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3280382290 0 0) 0x7fd16c003550 con 0x7fd1880a49a0 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd1880a0f60 con 0x7fd1880a49a0 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd178001770 con 0x7fd1880a49a0 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): dump: 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.444 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2358874249 0 0) 0x7fd188135070 con 0x7fd1880a41b0 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): my global_id is 6413 2024-05-08T15:53:31.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd16c003550 con 0x7fd1880a41b0 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1841558197 0 0) 0x7fd188134180 con 0x7fd188137940 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd188135070 con 0x7fd188137940 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd180001d50 con 0x7fd188137940 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient(hunting): dump: 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.446 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2406705299 0 0) 0x7fd1880a0f60 con 0x7fd1880a49a0 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.81:6791/0 conn(0x7fd1880a41b0 legacy=0x7fd1881cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.7:6789/0 conn(0x7fd188137940 legacy=0x7fd1881d4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 monclient: found mon.b 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1881343c0 con 0x7fd1880a49a0 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 5 monclient: authenticate success, global_id 95798 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 10 monclient: _renew_subs 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd1881336a0 con 0x7fd1880a49a0 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd178004f40 con 0x7fd1880a49a0 2024-05-08T15:53:31.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd178005430 con 0x7fd1880a49a0 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17dffb640 10 monclient: dump: 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.449 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.433+0000 7fd18f40f640 10 monclient: _renew_subs 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.433+0000 7fd18f40f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.432+0000 7fd17f7fe640 4 set_mon_vals no callback set 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.433+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fd188137730 con 0x7fd1880a49a0 2024-05-08T15:53:31.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.433+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd1780107b0 con 0x7fd1880a49a0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.433+0000 7fd18f40f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.433+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd1880753d0 con 0x7fd18809b5f0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.433+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(297..297 src has 1..297) ==== 3898+0+0 (unknown 2395490465 0 0) 0x7fd178047060 con 0x7fd1880a49a0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd18d1ab640 10 monclient: get_auth_request con 0x7fd16c03a1e0 auth_method 0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1454429660 0 0) 0x7fd1880753d0 con 0x7fd18809b5f0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 10 monclient(hunting): my global_id is 6220 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1880a0f60 con 0x7fd18809b5f0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 858008505 0 0) 0x7fd1880a0f60 con 0x7fd18809b5f0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd1880753d0 con 0x7fd18809b5f0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd180003cb0 con 0x7fd18809b5f0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 10 monclient: dump: 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.452 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 769566380 0 0) 0x7fd1880753d0 con 0x7fd18809b5f0 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.434+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd188135430 con 0x7fd18809b5f0 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.435+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fd1800049c0 con 0x7fd18809b5f0 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.435+0000 7fd17dffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.435+0000 7fd17dffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.435+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.81:6789/0 conn(0x7fd18809b5f0 legacy=0x7fd18809b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.435+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.81:6789/0 conn(0x7fd18809b5f0 legacy=0x7fd18809b9d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.441+0000 7fd18f40f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.441+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fd18806b2f0 con 0x7fd18809c0e0 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737117806 0 0) 0x7fd18806b2f0 con 0x7fd18809c0e0 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 10 monclient(hunting): my global_id is 6229 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1880753d0 con 0x7fd18809c0e0 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 200158079 0 0) 0x7fd1880753d0 con 0x7fd18809c0e0 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd18806b2f0 con 0x7fd18809c0e0 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd1800045e0 con 0x7fd18809c0e0 2024-05-08T15:53:31.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 10 monclient: dump: 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.456 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1507307000 0 0) 0x7fd18806b2f0 con 0x7fd18809c0e0 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.442+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd18809c4c0 con 0x7fd18809c0e0 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7695 (unknown 0 0 2714392269) 0x7fd180004b70 con 0x7fd18809c0e0 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd17dffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd17dffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.81:6789/0 conn(0x7fd18809c0e0 legacy=0x7fd1881d9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd17dffb640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.81:6789/0 conn(0x7fd18809c0e0 legacy=0x7fd1881d9c80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 403758, 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:31.458 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.459 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.460 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:31.461 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.462 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.463 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.464 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.465 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.466 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:31.467 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.468 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.469 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:31.470 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:31.471 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.472 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.473 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:31.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.475 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.476 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:53:31.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.477 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:31.477 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.81:6812/847627118 conn(0x7fd16c03a1e0 legacy=0x7fd16c03c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 10 monclient: shutdown 2024-05-08T15:53:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 >> v1:172.21.15.7:6790/0 conn(0x7fd1880a49a0 legacy=0x7fd188136e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18d9ac640 1 -- 172.21.15.7:0/4088061734 reap_dead start 2024-05-08T15:53:31.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 shutdown_connections 2024-05-08T15:53:31.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 >> 172.21.15.7:0/4088061734 conn(0x7fd1880a9fa0 msgr2=0x7fd188130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 shutdown_connections 2024-05-08T15:53:31.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.443+0000 7fd18f40f640 1 -- 172.21.15.7:0/4088061734 wait complete. 2024-05-08T15:53:31.480 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:31.480 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:53:31.480 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:53:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.588+0000 7f1997580640 10 monclient: get_monmap_and_config 2024-05-08T15:53:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.588+0000 7f1997580640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.588+0000 7f1997580640 10 monclient: monmap: 2024-05-08T15:53:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.584 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:31.589643+0000 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:31.589643+0000 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.585 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.589+0000 7f1997580640 1 Processor -- start 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.589+0000 7f1997580640 1 -- start start 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.589+0000 7f1997580640 10 monclient: init 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.589+0000 7f1997580640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.589+0000 7f1997580640 10 monclient: _add_conns ranks=[6,2,3,1,5,4,8,0,7] 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.589+0000 7f1997580640 10 monclient(hunting): picked mon.noname-d con 0x7f1990137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.589+0000 7f1997580640 10 monclient(hunting): picked mon.noname-b con 0x7f19900a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 10 monclient(hunting): picked mon.noname-g con 0x7f19900a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19900a2c70 con 0x7f19900a49a0 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19900a2f50 con 0x7f19900a41b0 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19900a1670 con 0x7f1990137940 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 10 monclient(hunting): authenticate will time out at 1219.704720s 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f199531c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f1990137940 0x7f1990137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35392/0 (socket says 172.21.15.7:35392) 2024-05-08T15:53:31.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f199531c640 1 -- 172.21.15.7:0/1374796877 learned_addr learned my addr 172.21.15.7:0/1374796877 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 38397419 0 0) 0x7f19900a1670 con 0x7f1990137940 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): my global_id is 124278 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f197c003520 con 0x7f1990137940 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2954366748 0 0) 0x7f19900a2c70 con 0x7f19900a49a0 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient(hunting): my global_id is 95807 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19900a1670 con 0x7f19900a49a0 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1837704766 0 0) 0x7f197c003520 con 0x7f1990137940 2024-05-08T15:53:31.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 >> v1:172.21.15.81:6790/0 conn(0x7f19900a41b0 legacy=0x7f1990140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 >> v1:172.21.15.7:6790/0 conn(0x7f19900a49a0 legacy=0x7f199013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 monclient: found mon.noname-d 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19900a39f0 con 0x7f1990137940 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 5 monclient: authenticate success, global_id 124278 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f198c0017b0 con 0x7f1990137940 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: dump: 2024-05-08T15:53:31.597 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.598 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f198c001a10 con 0x7f1990137940 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1997580640 4 set_mon_vals no callback set 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 1 -- 172.21.15.7:0/1374796877 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f198c004be0 con 0x7f1990137940 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.590+0000 7f1987fff640 10 monclient: dump: 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.599 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 10 monclient: shutdown 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 1 -- 172.21.15.7:0/1374796877 >> v1:172.21.15.7:6792/0 conn(0x7f1990137940 legacy=0x7f1990137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 1 -- 172.21.15.7:0/1374796877 shutdown_connections 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 1 -- 172.21.15.7:0/1374796877 >> 172.21.15.7:0/1374796877 conn(0x7f19900a9fa0 msgr2=0x7f19900aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 1 -- 172.21.15.7:0/1374796877 shutdown_connections 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 1 -- 172.21.15.7:0/1374796877 wait complete. 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.591+0000 7f1997580640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient: monmap: 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.601 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 1 Processor -- start 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 1 -- start start 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient: init 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient: _add_conns ranks=[2,8,3,1,0,6,7,5,4] 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient(hunting): picked mon.noname-b con 0x7f19900a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:31.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient(hunting): picked mon.noname-e con 0x7f19900a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient(hunting): picked mon.noname-g con 0x7f1990137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1990140be0 con 0x7f19900a41b0 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19900a1f30 con 0x7f1990137940 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19900a0f60 con 0x7f19900a49a0 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f1997580640 10 monclient(hunting): authenticate will time out at 1219.707522s 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f199531c640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f19900a41b0 0x7f19901351c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:52116/0 (socket says 172.21.15.7:52116) 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.592+0000 7f199531c640 1 -- 172.21.15.7:0/3574869930 learned_addr learned my addr 172.21.15.7:0/3574869930 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3082758927 0 0) 0x7f1990140be0 con 0x7f19900a41b0 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): my global_id is 95816 2024-05-08T15:53:31.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1974003520 con 0x7f19900a41b0 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 539814051 0 0) 0x7f19900a0f60 con 0x7f19900a49a0 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): my global_id is 95741 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1990140be0 con 0x7f19900a49a0 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 58887538 0 0) 0x7f19900a1f30 con 0x7f1990137940 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): my global_id is 76593 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19900a0f60 con 0x7f1990137940 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2187016161 0 0) 0x7f1974003520 con 0x7f19900a41b0 2024-05-08T15:53:31.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f19900a1f30 con 0x7f19900a41b0 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f198c004b20 con 0x7f19900a41b0 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): dump: 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.605 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1083154888 0 0) 0x7f1990140be0 con 0x7f19900a49a0 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1974003520 con 0x7f19900a49a0 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1980001570 con 0x7f19900a49a0 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:31.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient(hunting): dump: 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.607 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3463329706 0 0) 0x7f19900a0f60 con 0x7f1990137940 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1990140be0 con 0x7f1990137940 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 931655419 0 0) 0x7f19900a1f30 con 0x7f19900a41b0 2024-05-08T15:53:31.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.7:6793/0 conn(0x7f19900a49a0 legacy=0x7f19901358d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.81:6790/0 conn(0x7f1990137940 legacy=0x7f19901d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 monclient: found mon.b 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1990136b00 con 0x7f19900a41b0 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1997580640 5 monclient: authenticate success, global_id 95816 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1997580640 10 monclient: _renew_subs 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1997580640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1990139c70 con 0x7f19900a41b0 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f198c004e90 con 0x7f19900a41b0 2024-05-08T15:53:31.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f198c005380 con 0x7f19900a41b0 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1985ffb640 10 monclient: dump: 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.610 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1997580640 10 monclient: _renew_subs 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f19877fe640 4 set_mon_vals no callback set 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1997580640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.593+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f1990136f40 con 0x7f19900a41b0 2024-05-08T15:53:31.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.594+0000 7f1997580640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.594+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f19900753d0 con 0x7f1990135fe0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.594+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f198c010700 con 0x7f19900a41b0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.594+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(297..297 src has 1..297) ==== 3898+0+0 (unknown 2395490465 0 0) 0x7f198c046cf0 con 0x7f19900a41b0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1995b1d640 10 monclient: get_auth_request con 0x7f197403a300 auth_method 0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2705989791 0 0) 0x7f19900753d0 con 0x7f1990135fe0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient(hunting): my global_id is 76602 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19900a1f30 con 0x7f1990135fe0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1144578435 0 0) 0x7f19900a1f30 con 0x7f1990135fe0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f19900753d0 con 0x7f1990135fe0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1980003ca0 con 0x7f1990135fe0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient: dump: 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.613 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2362729981 0 0) 0x7f19900753d0 con 0x7f1990135fe0 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1990139ab0 con 0x7f1990135fe0 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f1980003fd0 con 0x7f1990135fe0 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.81:6790/0 conn(0x7f1990135fe0 legacy=0x7f199009db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.595+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.81:6790/0 conn(0x7f1990135fe0 legacy=0x7f199009db60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.602+0000 7f1997580640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:31.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.602+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f19900a5d60 con 0x7f199002d050 2024-05-08T15:53:31.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.602+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973056947 0 0) 0x7f19900a5d60 con 0x7f199002d050 2024-05-08T15:53:31.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient(hunting): my global_id is 76611 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19900753d0 con 0x7f199002d050 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2552771081 0 0) 0x7f19900753d0 con 0x7f199002d050 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f19900a5d60 con 0x7f199002d050 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1980004800 con 0x7f199002d050 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient: dump: 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1250275693 0 0) 0x7f19900a5d60 con 0x7f199002d050 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1990137660 con 0x7f199002d050 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1660166403) 0x7f1980005be0 con 0x7f199002d050 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.81:6790/0 conn(0x7f199002d050 legacy=0x7f199009e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.603+0000 7f1985ffb640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.81:6790/0 conn(0x7f199002d050 legacy=0x7f199009e270 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:31.619 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 403893, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 4, 2024-05-08T15:53:31.620 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.621 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.622 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.623 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.623 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.623 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.623 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.623 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.623 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.623 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:31.624 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:31.625 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.626 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.627 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.628 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.629 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.630 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:31.630 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:31.630 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.630 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.631 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.632 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.633 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.637 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.638 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.640 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.641 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.642 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:53:31.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.642 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:31.642 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.81:6812/847627118 conn(0x7f197403a300 legacy=0x7f197403c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 10 monclient: shutdown 2024-05-08T15:53:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 >> v1:172.21.15.7:6790/0 conn(0x7f19900a41b0 legacy=0x7f19901351c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1995b1d640 1 -- 172.21.15.7:0/3574869930 reap_dead start 2024-05-08T15:53:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1995b1d640 1 -- 172.21.15.7:0/3574869930 reap_dead start 2024-05-08T15:53:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 shutdown_connections 2024-05-08T15:53:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 >> 172.21.15.7:0/3574869930 conn(0x7f19900a9fa0 msgr2=0x7f19900a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 shutdown_connections 2024-05-08T15:53:31.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.604+0000 7f1997580640 1 -- 172.21.15.7:0/3574869930 wait complete. 2024-05-08T15:53:31.647 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:31.647 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:53:31.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:53:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.756+0000 7f92d1fab640 10 monclient: get_monmap_and_config 2024-05-08T15:53:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.756+0000 7f92d1fab640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.756+0000 7f92d1fab640 10 monclient: monmap: 2024-05-08T15:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:31.757678+0000 2024-05-08T15:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:31.757678+0000 2024-05-08T15:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.753 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.754 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.755 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.757+0000 7f92d1fab640 1 Processor -- start 2024-05-08T15:53:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.757+0000 7f92d1fab640 1 -- start start 2024-05-08T15:53:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.757+0000 7f92d1fab640 10 monclient: init 2024-05-08T15:53:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.757+0000 7f92d1fab640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.757+0000 7f92d1fab640 10 monclient: _add_conns ranks=[3,6,4,7,1,5,0,2,8] 2024-05-08T15:53:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 10 monclient(hunting): picked mon.noname-g con 0x7f92cc137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:31.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 10 monclient(hunting): picked mon.noname-d con 0x7f92cc0a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 10 monclient(hunting): picked mon.noname-c con 0x7f92cc0a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0a2c70 con 0x7f92cc137940 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0a2f50 con 0x7f92cc0a41b0 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0a1670 con 0x7f92cc0a49a0 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 10 monclient(hunting): authenticate will time out at 1219.872742s 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92caffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f92cc0a49a0 0x7f92cc13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35396/0 (socket says 172.21.15.7:35396) 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92caffd640 1 -- 172.21.15.7:0/963292666 learned_addr learned my addr 172.21.15.7:0/963292666 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92cbfff640 1 --1- 172.21.15.7:0/963292666 >> v1:172.21.15.7:6791/0 conn(0x7f92cc0a41b0 0x7f92cc140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51958/0 (socket says 172.21.15.7:51958) 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 400654770 0 0) 0x7f92cc0a1670 con 0x7f92cc0a49a0 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): my global_id is 124287 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b4003520 con 0x7f92cc0a49a0 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2643925850 0 0) 0x7f92cc0a2f50 con 0x7f92cc0a41b0 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): my global_id is 124267 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92cc0a1670 con 0x7f92cc0a41b0 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798579457 0 0) 0x7f92cc0a2c70 con 0x7f92cc137940 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient(hunting): my global_id is 76620 2024-05-08T15:53:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92cc0a2f50 con 0x7f92cc137940 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 414362448 0 0) 0x7f92b4003520 con 0x7f92cc0a49a0 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 >> v1:172.21.15.7:6791/0 conn(0x7f92cc0a41b0 legacy=0x7f92cc140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 >> v1:172.21.15.81:6790/0 conn(0x7f92cc137940 legacy=0x7f92cc137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 monclient: found mon.noname-d 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc0a39f0 con 0x7f92cc0a49a0 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92c00017b0 con 0x7f92cc0a49a0 2024-05-08T15:53:31.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 5 monclient: authenticate success, global_id 124287 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: dump: 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f92c0001a50 con 0x7f92cc0a49a0 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 1 -- 172.21.15.7:0/963292666 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92c0004be0 con 0x7f92cc0a49a0 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92d1fab640 4 set_mon_vals no callback set 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.758+0000 7f92ca7fc640 10 monclient: dump: 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 10 monclient: shutdown 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 1 -- 172.21.15.7:0/963292666 >> v1:172.21.15.7:6792/0 conn(0x7f92cc0a49a0 legacy=0x7f92cc13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 1 -- 172.21.15.7:0/963292666 shutdown_connections 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 1 -- 172.21.15.7:0/963292666 >> 172.21.15.7:0/963292666 conn(0x7f92cc0a9fa0 msgr2=0x7f92cc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 1 -- 172.21.15.7:0/963292666 shutdown_connections 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 1 -- 172.21.15.7:0/963292666 wait complete. 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.759+0000 7f92d1fab640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.760+0000 7f92d1fab640 10 monclient: monmap: 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.778 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.760+0000 7f92d1fab640 1 Processor -- start 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.760+0000 7f92d1fab640 1 -- start start 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.760+0000 7f92d1fab640 10 monclient: init 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.760+0000 7f92d1fab640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.760+0000 7f92d1fab640 10 monclient: _add_conns ranks=[8,7,2,6,4,5,3,1,0] 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 10 monclient(hunting): picked mon.noname-e con 0x7f92cc0a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 10 monclient(hunting): picked mon.noname-i con 0x7f92cc0a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 10 monclient(hunting): picked mon.noname-b con 0x7f92cc137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc137350 con 0x7f92cc137940 2024-05-08T15:53:31.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc0a0f60 con 0x7f92cc0a49a0 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92cc135890 con 0x7f92cc0a41b0 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92cb7fe640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f92cc0a41b0 0x7f92cc141280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35612/0 (socket says 172.21.15.7:35612) 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92cb7fe640 1 -- 172.21.15.7:0/3184752443 learned_addr learned my addr 172.21.15.7:0/3184752443 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92d1fab640 10 monclient(hunting): authenticate will time out at 1219.875799s 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1304131311 0 0) 0x7f92cc135890 con 0x7f92cc0a41b0 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): my global_id is 95750 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92b8003520 con 0x7f92cc0a41b0 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1846070390 0 0) 0x7f92c4001d60 con 0x7f92cc137940 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): my global_id is 95825 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92cc135890 con 0x7f92cc137940 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2831615871 0 0) 0x7f92cc0a0f60 con 0x7f92cc0a49a0 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): my global_id is 114946 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92c4001d60 con 0x7f92cc0a49a0 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1598488412 0 0) 0x7f92b8003520 con 0x7f92cc0a41b0 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92cc0a0f60 con 0x7f92cc0a41b0 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92bc001ad0 con 0x7f92cc0a41b0 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): dump: 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.782 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1729849187 0 0) 0x7f92cc135890 con 0x7f92cc137940 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92b8003520 con 0x7f92cc137940 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92c4001ad0 con 0x7f92cc137940 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): dump: 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.783 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3341705442 0 0) 0x7f92c4001d60 con 0x7f92cc0a49a0 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92cc135890 con 0x7f92cc0a49a0 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92c0004b20 con 0x7f92cc0a49a0 2024-05-08T15:53:31.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient(hunting): dump: 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.785 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4011667851 0 0) 0x7f92cc0a0f60 con 0x7f92cc0a41b0 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.81:6792/0 conn(0x7f92cc0a49a0 legacy=0x7f92cc1d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.7:6790/0 conn(0x7f92cc137940 legacy=0x7f92cc1d4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 1 monclient: found mon.e 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.761+0000 7f92b3fff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92cc1ccd20 con 0x7f92cc0a41b0 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 5 monclient: authenticate success, global_id 95750 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 10 monclient: _renew_subs 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f92cc141990 con 0x7f92cc0a41b0 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 10 monclient: _renew_subs 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f92cc1367f0 con 0x7f92cc0a41b0 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f92bc0038c0 con 0x7f92cc0a41b0 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92bc003db0 con 0x7f92cc0a41b0 2024-05-08T15:53:31.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92c9ffb640 4 set_mon_vals no callback set 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92b3fff640 10 monclient: dump: 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.788 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.762+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f92ac0065c0 con 0x7f92ac002860 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f92bc003f50 con 0x7f92cc0a41b0 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(297..297 src has 1..297) ==== 3898+0+0 (unknown 2395490465 0 0) 0x7f92bc046c80 con 0x7f92cc0a41b0 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674625903 0 0) 0x7f92ac0065c0 con 0x7f92ac002860 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 10 monclient(hunting): my global_id is 6422 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92cc0a0f60 con 0x7f92ac002860 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92cbfff640 10 monclient: get_auth_request con 0x7f92b803a3d0 auth_method 0 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3540074822 0 0) 0x7f92cc0a0f60 con 0x7f92ac002860 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92ac0065c0 con 0x7f92ac002860 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92c0001d40 con 0x7f92ac002860 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 10 monclient: dump: 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.790 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 733335829 0 0) 0x7f92ac0065c0 con 0x7f92ac002860 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.763+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f92ac00a150 con 0x7f92ac002860 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.764+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f92c0007080 con 0x7f92ac002860 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.764+0000 7f92b3fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.764+0000 7f92b3fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.764+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.81:6791/0 conn(0x7f92ac002860 legacy=0x7f92ac004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.764+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.81:6791/0 conn(0x7f92ac002860 legacy=0x7f92ac004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.770+0000 7f92d1fab640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.770+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f92ac009e20 con 0x7f92ac0090e0 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3936340377 0 0) 0x7f92ac009e20 con 0x7f92ac0090e0 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 10 monclient(hunting): my global_id is 6431 2024-05-08T15:53:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f92ac0065c0 con 0x7f92ac0090e0 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 639168770 0 0) 0x7f92ac0065c0 con 0x7f92ac0090e0 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f92ac009e20 con 0x7f92ac0090e0 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f92c0005630 con 0x7f92ac0090e0 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 10 monclient: dump: 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.793 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2621101575 0 0) 0x7f92ac009e20 con 0x7f92ac0090e0 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.771+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f92ac00c390 con 0x7f92ac0090e0 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 2503670901) 0x7f92c0009fd0 con 0x7f92ac0090e0 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92b3fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92b3fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.81:6791/0 conn(0x7f92ac0090e0 legacy=0x7f92ac009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92b3fff640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.81:6791/0 conn(0x7f92ac0090e0 legacy=0x7f92ac009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 404077, 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:31.795 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 5, 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.796 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.797 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:31.798 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:31.799 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:31.800 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.801 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.802 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.803 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.804 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.805 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:31.806 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:31.807 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.808 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:31.809 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.810 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.811 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.812 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.813 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:31.814 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38908]: starting. 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38908]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38868 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[38908]: shutting down. 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.81:6812/847627118 conn(0x7f92b803a3d0 legacy=0x7f92b803c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 10 monclient: shutdown 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 >> v1:172.21.15.7:6793/0 conn(0x7f92cc0a41b0 legacy=0x7f92cc141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92cbfff640 1 -- 172.21.15.7:0/3184752443 reap_dead start 2024-05-08T15:53:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92cbfff640 1 -- 172.21.15.7:0/3184752443 reap_dead start 2024-05-08T15:53:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 shutdown_connections 2024-05-08T15:53:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 >> 172.21.15.7:0/3184752443 conn(0x7f92cc0a9fa0 msgr2=0x7f92cc0a8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 shutdown_connections 2024-05-08T15:53:31.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.772+0000 7f92d1fab640 1 -- 172.21.15.7:0/3184752443 wait complete. 2024-05-08T15:53:31.818 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:53:31.819 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:31.819 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:53:31.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:53:31.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.938+0000 7ff0c7de7640 10 monclient: get_monmap_and_config 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.938+0000 7ff0c7de7640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.938+0000 7ff0c7de7640 10 monclient: monmap: 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:31.939074+0000 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:31.939074+0000 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.934 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.935 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.938+0000 7ff0c7de7640 1 Processor -- start 2024-05-08T15:53:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.938+0000 7ff0c7de7640 1 -- start start 2024-05-08T15:53:31.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.938+0000 7ff0c7de7640 10 monclient: init 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 10 monclient: _add_conns ranks=[5,8,1,0,4,6,3,7,2] 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 10 monclient(hunting): picked mon.noname-h con 0x7ff0c00a53a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 10 monclient(hunting): picked mon.noname-e con 0x7ff0c00a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 10 monclient(hunting): picked mon.noname-f con 0x7ff0c013e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c00a1660 con 0x7ff0c013e310 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c00a28f0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c00a2bd0 con 0x7ff0c00a5780 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c7de7640 10 monclient(hunting): authenticate will time out at 1220.054098s 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c5382640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7ff0c00a5780 0x7ff0c013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:35624/0 (socket says 172.21.15.7:35624) 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c5382640 1 -- 172.21.15.7:0/3911114086 learned_addr learned my addr 172.21.15.7:0/3911114086 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3110519829 0 0) 0x7ff0c00a2bd0 con 0x7ff0c00a5780 2024-05-08T15:53:31.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c4b81640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c4b81640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c4b81640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c4b81640 10 monclient(hunting): my global_id is 95777 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.939+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0a8003520 con 0x7ff0c00a5780 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3443574600 0 0) 0x7ff0a8003520 con 0x7ff0c00a5780 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 >> v1:172.21.15.81:6791/0 conn(0x7ff0c00a53a0 legacy=0x7ff0c00a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 >> v1:172.21.15.81:6789/0 conn(0x7ff0c013e310 legacy=0x7ff0c0140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 monclient: found mon.noname-e 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:53:31.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c00a6730 con 0x7ff0c00a5780 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 5 monclient: authenticate success, global_id 95777 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b4001b30 con 0x7ff0c00a5780 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: dump: 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.947 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff0b4001e90 con 0x7ff0c00a5780 2024-05-08T15:53:31.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 1 -- 172.21.15.7:0/3911114086 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b4004f60 con 0x7ff0c00a5780 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 4 set_mon_vals no callback set 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c4b81640 10 monclient: dump: 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.949 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.950 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 10 monclient: shutdown 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 1 -- 172.21.15.7:0/3911114086 >> v1:172.21.15.7:6793/0 conn(0x7ff0c00a5780 legacy=0x7ff0c013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 1 -- 172.21.15.7:0/3911114086 shutdown_connections 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 1 -- 172.21.15.7:0/3911114086 >> 172.21.15.7:0/3911114086 conn(0x7ff0c00a9fa0 msgr2=0x7ff0c00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 1 -- 172.21.15.7:0/3911114086 shutdown_connections 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.940+0000 7ff0c7de7640 1 -- 172.21.15.7:0/3911114086 wait complete. 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.941+0000 7ff0c7de7640 10 monclient: build_initial_monmap 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.941+0000 7ff0c7de7640 10 monclient: monmap: 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:31.951 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.941+0000 7ff0c7de7640 1 Processor -- start 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.941+0000 7ff0c7de7640 1 -- start start 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.941+0000 7ff0c7de7640 10 monclient: init 2024-05-08T15:53:31.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 10 monclient: _add_conns ranks=[6,3,0,1,5,8,2,4,7] 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 10 monclient(hunting): picked mon.noname-d con 0x7ff0c00a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 10 monclient(hunting): picked mon.noname-g con 0x7ff0c00a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 10 monclient(hunting): picked mon.noname-a con 0x7ff0c013e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c013a1f0 con 0x7ff0c013e310 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c013e120 con 0x7ff0c00a5780 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0c0139af0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c7de7640 10 monclient(hunting): authenticate will time out at 1220.056825s 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c5b83640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7ff0c00a53a0 0x7ff0c013da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35406/0 (socket says 172.21.15.7:35406) 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0c5b83640 1 -- 172.21.15.7:0/1679126818 learned_addr learned my addr 172.21.15.7:0/1679126818 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:31.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2638625181 0 0) 0x7ff0c0139af0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): my global_id is 124323 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0ac003520 con 0x7ff0c00a53a0 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2286075073 0 0) 0x7ff0c013e120 con 0x7ff0c00a5780 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): my global_id is 76629 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0c0139af0 con 0x7ff0c00a5780 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000294425 0 0) 0x7ff0c013a1f0 con 0x7ff0c013e310 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): my global_id is 95688 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0c013e120 con 0x7ff0c013e310 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1529227160 0 0) 0x7ff0ac003520 con 0x7ff0c00a53a0 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0c013a1f0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b8001b60 con 0x7ff0c00a53a0 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): dump: 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.955 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2372066560 0 0) 0x7ff0c0139af0 con 0x7ff0c00a5780 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0ac003520 con 0x7ff0c00a5780 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1903794949 0 0) 0x7ff0c013e120 con 0x7ff0c013e310 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0c0139af0 con 0x7ff0c013e310 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b4001af0 con 0x7ff0c00a5780 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.942+0000 7ff0be7fc640 10 monclient(hunting): dump: 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.957 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b00017e0 con 0x7ff0c013e310 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient(hunting): dump: 2024-05-08T15:53:31.958 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3176718220 0 0) 0x7ff0c013a1f0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.81:6790/0 conn(0x7ff0c00a5780 legacy=0x7ff0c01d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.7:6789/0 conn(0x7ff0c013e310 legacy=0x7ff0c01d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 monclient: found mon.d 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c013bc40 con 0x7ff0c00a53a0 2024-05-08T15:53:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: _finish_auth 0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 5 monclient: authenticate success, global_id 124323 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 10 monclient: _renew_subs 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c013bf20 con 0x7ff0c00a53a0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 10 monclient: _renew_subs 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7ff0c013aee0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff0b80038c0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b8003e30 con 0x7ff0c00a53a0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0bffff640 4 set_mon_vals no callback set 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0be7fc640 10 monclient: dump: 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.943+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff0a00065c0 con 0x7ff0a0002860 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7ff0b8003fd0 con 0x7ff0c00a53a0 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(298..298 src has 1..298) ==== 3506+0+0 (unknown 3354227398 0 0) 0x7ff0b8046d00 con 0x7ff0c00a53a0 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194443929 0 0) 0x7ff0a00065c0 con 0x7ff0a0002860 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 10 monclient(hunting): my global_id is 114964 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0c013a1f0 con 0x7ff0a0002860 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0c6384640 10 monclient: get_auth_request con 0x7ff0ac03a4c0 auth_method 0 2024-05-08T15:53:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1339053575 0 0) 0x7ff0c013a1f0 con 0x7ff0a0002860 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0a00065c0 con 0x7ff0a0002860 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b4001e90 con 0x7ff0a0002860 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 10 monclient: dump: 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3090609690 0 0) 0x7ff0a00065c0 con 0x7ff0a0002860 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.944+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0a000a150 con 0x7ff0a0002860 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.945+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff0b40076f0 con 0x7ff0a0002860 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.945+0000 7ff0be7fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.945+0000 7ff0be7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.945+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.81:6792/0 conn(0x7ff0a0002860 legacy=0x7ff0a0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.945+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.81:6792/0 conn(0x7ff0a0002860 legacy=0x7ff0a0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.951+0000 7ff0c7de7640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.951+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff0a0009e40 con 0x7ff0a0009100 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1796655457 0 0) 0x7ff0a0009e40 con 0x7ff0a0009100 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient(hunting): my global_id is 114973 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0a00065c0 con 0x7ff0a0009100 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4114268588 0 0) 0x7ff0a00065c0 con 0x7ff0a0009100 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff0a0009e40 con 0x7ff0a0009100 2024-05-08T15:53:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff0b4005f00 con 0x7ff0a0009100 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient: dump: 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1189614191 0 0) 0x7ff0a0009e40 con 0x7ff0a0009100 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff0a000c3d0 con 0x7ff0a0009100 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 577780035) 0x7ff0b40072a0 con 0x7ff0a0009100 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.952+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.81:6792/0 conn(0x7ff0a0009100 legacy=0x7ff0a0009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0be7fc640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.81:6792/0 conn(0x7ff0a0009100 legacy=0x7ff0a0009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:31.969 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 50, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 146817, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:31.970 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 5, 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.971 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.972 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:31.973 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:31.974 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.975 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.976 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.977 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.978 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.979 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.980 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.981 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:31.982 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.984 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.985 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.986 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:31.987 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:31.988 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:31.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:31.988 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:31.988 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.81:6812/847627118 conn(0x7ff0ac03a4c0 legacy=0x7ff0ac03c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 10 monclient: shutdown 2024-05-08T15:53:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 >> v1:172.21.15.7:6792/0 conn(0x7ff0c00a53a0 legacy=0x7ff0c013da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c6384640 1 -- 172.21.15.7:0/1679126818 reap_dead start 2024-05-08T15:53:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c6384640 1 -- 172.21.15.7:0/1679126818 reap_dead start 2024-05-08T15:53:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 shutdown_connections 2024-05-08T15:53:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 >> 172.21.15.7:0/1679126818 conn(0x7ff0c00a9fa0 msgr2=0x7ff0c0136e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 shutdown_connections 2024-05-08T15:53:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:31.953+0000 7ff0c7de7640 1 -- 172.21.15.7:0/1679126818 wait complete. 2024-05-08T15:53:31.990 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:53:31.990 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['e'] 2024-05-08T15:53:31.990 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['a'] 2024-05-08T15:53:31.990 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.e 2024-05-08T15:53:31.991 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-05-08T15:53:31.991 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-05-08T15:53:31.991 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:53:32.003 INFO:tasks.ceph.mon.e:Stopped 2024-05-08T15:53:32.003 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-05-08T15:53:32.003 INFO:tasks.ceph.mon.a:Sent signal 19 2024-05-08T15:53:32.003 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-05-08T15:53:47.004 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-05-08T15:53:47.004 INFO:tasks.ceph.mon.a:Sent signal 18 2024-05-08T15:53:47.005 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 8 2024-05-08T15:53:47.005 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:53:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.124+0000 7f293146a640 10 monclient: get_monmap_and_config 2024-05-08T15:53:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.124+0000 7f293146a640 10 monclient: build_initial_monmap 2024-05-08T15:53:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.124+0000 7f293146a640 10 monclient: monmap: 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:47.125775+0000 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:47.125775+0000 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:47.122 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 1 Processor -- start 2024-05-08T15:53:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 1 -- start start 2024-05-08T15:53:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 10 monclient: init 2024-05-08T15:53:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 10 monclient: _add_conns ranks=[3,8,4,2,7,0,1,6,5] 2024-05-08T15:53:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 10 monclient(hunting): picked mon.noname-g con 0x7f292c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 10 monclient(hunting): picked mon.noname-e con 0x7f292c0a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 10 monclient(hunting): picked mon.noname-c con 0x7f292c0a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.125+0000 7f293146a640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c0a2c70 con 0x7f292c137940 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.126+0000 7f293146a640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c0a2f50 con 0x7f292c0a41b0 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.126+0000 7f293146a640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c0a1670 con 0x7f292c0a49a0 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.126+0000 7f293146a640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.126+0000 7f293146a640 10 monclient(hunting): authenticate will time out at 1235.240626s 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.126+0000 7f292a7fc640 1 -- >> v1:172.21.15.7:6793/0 conn(0x7f292c0a49a0 legacy=0x7f292c13cee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.126+0000 7f292b7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f292c0a41b0 0x7f292c140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:52300/0 (socket says 172.21.15.7:52300) 2024-05-08T15:53:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.126+0000 7f292b7fe640 1 -- 172.21.15.7:0/3278813095 learned_addr learned my addr 172.21.15.7:0/3278813095 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:47.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.327+0000 7f292a7fc640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.7:6793/0 conn(0x7f292c0a49a0 legacy=0x7f292c13cee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:53:47.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:47.728+0000 7f292a7fc640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.7:6793/0 conn(0x7f292c0a49a0 legacy=0x7f292c13cee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:53:48.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:48.529+0000 7f292a7fc640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.7:6793/0 conn(0x7f292c0a49a0 legacy=0x7f292c13cee0 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:53:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient(hunting): tick 2024-05-08T15:53:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient(hunting): _check_auth_tickets 2024-05-08T15:53:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 1 monclient(hunting): continuing hunt 2024-05-08T15:53:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient(hunting): _reopen_session rank -1 2024-05-08T15:53:50.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.7:6793/0 conn(0x7f292c0a49a0 legacy=0x7f292c13cee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.7:6791/0 conn(0x7f292c0a41b0 legacy=0x7f292c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.81:6790/0 conn(0x7f292c137940 legacy=0x7f292c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient: _add_conns ranks=[7,0,6,1,5,2] 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient(hunting): picked mon.noname-i con 0x7f29180037c0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient(hunting): picked mon.noname-a con 0x7f2918007450 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient(hunting): picked mon.noname-d con 0x7f291800b1e0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 1 -- 172.21.15.7:0/3278813095 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c0a39f0 con 0x7f2918007450 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 1 -- 172.21.15.7:0/3278813095 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c0a1670 con 0x7f291800b1e0 2024-05-08T15:53:50.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 1 -- 172.21.15.7:0/3278813095 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2918011000 con 0x7f29180037c0 2024-05-08T15:53:50.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:50.126+0000 7f2928ff9640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328710754 0 0) 0x7f2918011000 con 0x7f29180037c0 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): my global_id is 114982 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2914003520 con 0x7f29180037c0 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2343087992 0 0) 0x7f292c0a1670 con 0x7f291800b1e0 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient(hunting): my global_id is 124332 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2918011000 con 0x7f291800b1e0 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1547695237 0 0) 0x7f2914003520 con 0x7f29180037c0 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.7:6792/0 conn(0x7f291800b1e0 legacy=0x7f291800d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.7:6789/0 conn(0x7f2918007450 legacy=0x7f2918009910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 monclient: found mon.noname-i 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f292b7fe640 1 -- 172.21.15.7:0/3278813095 reap_dead start 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient: _send_mon_message to mon.noname-i at v1:172.21.15.81:6792/0 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2918012290 con 0x7f29180037c0 2024-05-08T15:53:52.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f291c0016f0 con 0x7f29180037c0 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient: got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f2929ffb640 10 monclient: dump: 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.018 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f293146a640 5 monclient: authenticate success, global_id 114982 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.019+0000 7f293146a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 <== mon.7 v1:172.21.15.81:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f291c001950 con 0x7f29180037c0 2024-05-08T15:53:52.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f2929ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f2929ffb640 1 -- 172.21.15.7:0/3278813095 <== mon.7 v1:172.21.15.81:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f291c003950 con 0x7f29180037c0 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f293146a640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f293146a640 4 set_mon_vals no callback set 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f2929ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f2929ffb640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.020+0000 7f2929ffb640 10 monclient: dump: 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.020 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.021 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.021+0000 7f293146a640 10 monclient: shutdown 2024-05-08T15:53:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.021+0000 7f293146a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.021+0000 7f293146a640 1 -- 172.21.15.7:0/3278813095 >> v1:172.21.15.81:6792/0 conn(0x7f29180037c0 legacy=0x7f2918005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.021+0000 7f293146a640 1 -- 172.21.15.7:0/3278813095 shutdown_connections 2024-05-08T15:53:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.021+0000 7f293146a640 1 -- 172.21.15.7:0/3278813095 >> 172.21.15.7:0/3278813095 conn(0x7f292c0a9fa0 msgr2=0x7f292c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.021+0000 7f293146a640 1 -- 172.21.15.7:0/3278813095 shutdown_connections 2024-05-08T15:53:52.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.021+0000 7f293146a640 1 -- 172.21.15.7:0/3278813095 wait complete. 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.022+0000 7f293146a640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.022+0000 7f293146a640 10 monclient: monmap: 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.027 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.023+0000 7f293146a640 1 Processor -- start 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.023+0000 7f293146a640 1 -- start start 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.023+0000 7f293146a640 10 monclient: init 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 10 monclient: _add_conns ranks=[0,1,3,4,7,5,2,8,6] 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 10 monclient(hunting): picked mon.noname-a con 0x7f29180037c0 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:52.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 10 monclient(hunting): picked mon.noname-f con 0x7f292c125090 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 10 monclient(hunting): picked mon.noname-g con 0x7f292c121cc0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c122980 con 0x7f29180037c0 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c1234e0 con 0x7f292c125090 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f292c0a0f60 con 0x7f292c121cc0 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f293146a640 10 monclient(hunting): authenticate will time out at 1240.138863s 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f292affd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f29180037c0 0x7f292c1ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35862/0 (socket says 172.21.15.7:35862) 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f292affd640 1 -- 172.21.15.7:0/2238424870 learned_addr learned my addr 172.21.15.7:0/2238424870 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2469550491 0 0) 0x7f292c1234e0 con 0x7f292c125090 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f29137fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f29137fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f29137fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f29137fe640 10 monclient(hunting): my global_id is 6256 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f291800efd0 con 0x7f292c125090 2024-05-08T15:53:52.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.024+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3313856372 0 0) 0x7f291800efd0 con 0x7f292c125090 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f292c1234e0 con 0x7f292c125090 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2920004410 con 0x7f292c125090 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient(hunting): dump: 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.030 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.031 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.031 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3340336626 0 0) 0x7f292c1234e0 con 0x7f292c125090 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 >> v1:172.21.15.81:6790/0 conn(0x7f292c121cc0 legacy=0x7f292c1d52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 >> v1:172.21.15.7:6789/0 conn(0x7f29180037c0 legacy=0x7f292c1ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 monclient: found mon.f 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c0a1f30 con 0x7f292c125090 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f293146a640 5 monclient: authenticate success, global_id 6256 2024-05-08T15:53:52.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f293146a640 10 monclient: _renew_subs 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f293146a640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f292c124990 con 0x7f292c125090 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f293146a640 10 monclient: _renew_subs 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f293146a640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f292c1cd2c0 con 0x7f292c125090 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2920004970 con 0x7f292c125090 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2920004e60 con 0x7f292c125090 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29137fe640 10 monclient: dump: 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.033 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.025+0000 7f29297fa640 4 set_mon_vals no callback set 2024-05-08T15:53:52.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.026+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f29200060b0 con 0x7f292c125090 2024-05-08T15:53:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.026+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(299..299 src has 1..299) ==== 3898+0+0 (unknown 1495620140 0 0) 0x7f2920046ef0 con 0x7f292c125090 2024-05-08T15:53:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.026+0000 7f292affd640 10 monclient: get_auth_request con 0x7f291803afc0 auth_method 0 2024-05-08T15:53:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.027+0000 7f293146a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.027+0000 7f293146a640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.027+0000 7f293146a640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:53:52.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.027+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292c0753d0 con 0x7f292c125090 2024-05-08T15:53:52.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.032+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f2920019af0 con 0x7f292c125090 2024-05-08T15:53:52.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.032+0000 7f29137fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.032+0000 7f29137fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:52.169 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38980]: starting. 2024-05-08T15:53:52.170 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38980]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38977 2024-05-08T15:53:52.170 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38980]: created object 0... 2024-05-08T15:53:52.170 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38980]: created object 25... 2024-05-08T15:53:52.170 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38980]: created object 49... 2024-05-08T15:53:52.170 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38980]: finishing. 2024-05-08T15:53:52.170 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[38980]: shutting down. 2024-05-08T15:53:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.311+0000 7f293146a640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:53:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.311+0000 7f293146a640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:53:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.311+0000 7f293146a640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:53:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.311+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f292c06b2f0 con 0x7f292c125090 2024-05-08T15:53:52.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.311+0000 7f29137fe640 1 -- 172.21.15.7:0/2238424870 <== mon.1 v1:172.21.15.81:6789/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2696 (unknown 254746657 0 3804459286) 0x7f292001e3b0 con 0x7f292c125090 2024-05-08T15:53:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.311+0000 7f29137fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:53:52.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.311+0000 7f29137fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:52.308 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":56,"quorum":[0,1,2,3,4,5,6,7],"quorum_names":["a","f","b","g","c","h","d","i"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:53:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 >> v1:172.21.15.81:6812/847627118 conn(0x7f291803afc0 legacy=0x7f291803d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 10 monclient: shutdown 2024-05-08T15:53:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 >> v1:172.21.15.81:6789/0 conn(0x7f292c125090 legacy=0x7f292c1d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 shutdown_connections 2024-05-08T15:53:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 >> 172.21.15.7:0/2238424870 conn(0x7f292c0a9fa0 msgr2=0x7f292c0a8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 shutdown_connections 2024-05-08T15:53:52.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.313+0000 7f293146a640 1 -- 172.21.15.7:0/2238424870 wait complete. 2024-05-08T15:53:52.321 INFO:tasks.mon_thrash.ceph_manager:quorum is size 8 2024-05-08T15:53:52.321 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon a 2024-05-08T15:53:52.321 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:53:52.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.432+0000 7fcb085bd640 10 monclient: get_monmap_and_config 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.432+0000 7fcb085bd640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.432+0000 7fcb085bd640 10 monclient: monmap: 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:52.433535+0000 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:52.433535+0000 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.429 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 1 Processor -- start 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 1 -- start start 2024-05-08T15:53:52.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient: init 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient: _add_conns ranks=[2,7,6,4,1,5,0,3,8] 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient(hunting): picked mon.noname-b con 0x7fcb00137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient(hunting): picked mon.noname-i con 0x7fcb000a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient(hunting): picked mon.noname-d con 0x7fcb000a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb000a2c70 con 0x7fcb00137940 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb000a2f50 con 0x7fcb000a41b0 2024-05-08T15:53:52.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb000a1670 con 0x7fcb000a49a0 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb085bd640 10 monclient(hunting): authenticate will time out at 1240.548456s 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb06359640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fcb00137940 0x7fcb00137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43612/0 (socket says 172.21.15.7:43612) 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.433+0000 7fcb06359640 1 -- 172.21.15.7:0/1166414005 learned_addr learned my addr 172.21.15.7:0/1166414005 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554237049 0 0) 0x7fcb000a2c70 con 0x7fcb00137940 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): my global_id is 95834 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcae8003520 con 0x7fcb00137940 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3328333962 0 0) 0x7fcb000a2f50 con 0x7fcb000a41b0 2024-05-08T15:53:52.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): my global_id is 124341 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb000a2c70 con 0x7fcb000a41b0 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4166936867 0 0) 0x7fcb000a1670 con 0x7fcb000a49a0 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient(hunting): my global_id is 114991 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb000a2f50 con 0x7fcb000a49a0 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2335722417 0 0) 0x7fcb000a2c70 con 0x7fcb000a41b0 2024-05-08T15:53:52.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 >> v1:172.21.15.81:6792/0 conn(0x7fcb000a49a0 legacy=0x7fcb0013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 >> v1:172.21.15.7:6790/0 conn(0x7fcb00137940 legacy=0x7fcb00137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 monclient: found mon.noname-d 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb000a39f0 con 0x7fcb000a41b0 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb085bd640 5 monclient: authenticate success, global_id 124341 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf8001b10 con 0x7fcb000a41b0 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb085bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: dump: 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.449 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb085bd640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:52.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcaf8001df0 con 0x7fcb000a41b0 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb085bd640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb085bd640 4 set_mon_vals no callback set 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 1 -- 172.21.15.7:0/1166414005 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf8003cd0 con 0x7fcb000a41b0 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.434+0000 7fcb05357640 10 monclient: dump: 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.451 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.452 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 10 monclient: shutdown 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 1 -- 172.21.15.7:0/1166414005 >> v1:172.21.15.7:6792/0 conn(0x7fcb000a41b0 legacy=0x7fcb00140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 1 -- 172.21.15.7:0/1166414005 shutdown_connections 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 1 -- 172.21.15.7:0/1166414005 >> 172.21.15.7:0/1166414005 conn(0x7fcb000a9fa0 msgr2=0x7fcb000aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 1 -- 172.21.15.7:0/1166414005 shutdown_connections 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 1 -- 172.21.15.7:0/1166414005 wait complete. 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.435+0000 7fcb085bd640 10 monclient: monmap: 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.454 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 1 Processor -- start 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 1 -- start start 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient: init 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient: _add_conns ranks=[4,0,5,8,2,6,7,3,1] 2024-05-08T15:53:52.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient(hunting): picked mon.noname-c con 0x7fcb000a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient(hunting): picked mon.noname-a con 0x7fcb000a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient(hunting): picked mon.noname-h con 0x7fcb00137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb00138c50 con 0x7fcb000a49a0 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb001cda40 con 0x7fcb000a41b0 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb001366f0 con 0x7fcb00137940 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb085bd640 10 monclient(hunting): authenticate will time out at 1240.551255s 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb06359640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fcb000a41b0 0x7fcb001ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:52312/0 (socket says 172.21.15.7:52312) 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcb06359640 1 -- 172.21.15.7:0/1035837860 learned_addr learned my addr 172.21.15.7:0/1035837860 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1372146017 0 0) 0x7fcb001cda40 con 0x7fcb000a41b0 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): my global_id is 124276 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcaec003520 con 0x7fcb000a41b0 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832152268 0 0) 0x7fcb00138c50 con 0x7fcb000a49a0 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.436+0000 7fcafeffd640 10 monclient(hunting): my global_id is 95697 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb001cda40 con 0x7fcb000a49a0 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1092559207 0 0) 0x7fcb001366f0 con 0x7fcb00137940 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): my global_id is 6476 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb00138c50 con 0x7fcb00137940 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3218682765 0 0) 0x7fcaec003520 con 0x7fcb000a41b0 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcb001366f0 con 0x7fcb000a41b0 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2918746013 0 0) 0x7fcb001cda40 con 0x7fcb000a49a0 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcaec003520 con 0x7fcb000a49a0 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf00019e0 con 0x7fcb000a41b0 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): dump: 2024-05-08T15:53:52.458 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.459 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf4001990 con 0x7fcb000a49a0 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): dump: 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.460 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 569002511 0 0) 0x7fcb00138c50 con 0x7fcb00137940 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcb001cda40 con 0x7fcb00137940 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf8003cb0 con 0x7fcb00137940 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient(hunting): dump: 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.462 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.463 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 171403530 0 0) 0x7fcb001366f0 con 0x7fcb000a41b0 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.81:6791/0 conn(0x7fcb00137940 legacy=0x7fcb001d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.7:6789/0 conn(0x7fcb000a49a0 legacy=0x7fcb001cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 monclient: found mon.c 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb000a1f30 con 0x7fcb000a41b0 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb085bd640 5 monclient: authenticate success, global_id 124276 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb085bd640 10 monclient: _renew_subs 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb085bd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fcb000a0f60 con 0x7fcb000a41b0 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb085bd640 10 monclient: _renew_subs 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb085bd640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7fcb00136470 con 0x7fcb000a41b0 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fcaf0004a90 con 0x7fcb000a41b0 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf0001c10 con 0x7fcb000a41b0 2024-05-08T15:53:52.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcb04b56640 4 set_mon_vals no callback set 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.437+0000 7fcafeffd640 10 monclient: dump: 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.466 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.467 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcb085bd640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fcaf0010160 con 0x7fcb000a41b0 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fcae00065c0 con 0x7fcae0002860 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(300..300 src has 1..300) ==== 3898+0+0 (unknown 3456018804 0 0) 0x7fcaf0046e30 con 0x7fcb000a41b0 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2846672150 0 0) 0x7fcae00065c0 con 0x7fcae0002860 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 10 monclient(hunting): my global_id is 95706 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb001366f0 con 0x7fcae0002860 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.438+0000 7fcb06b5a640 10 monclient: get_auth_request con 0x7fcaec03a410 auth_method 0 2024-05-08T15:53:52.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4289561827 0 0) 0x7fcb001366f0 con 0x7fcae0002860 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcae00065c0 con 0x7fcae0002860 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf4003c00 con 0x7fcae0002860 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 10 monclient: dump: 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.469 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2526807608 0 0) 0x7fcae00065c0 con 0x7fcae0002860 2024-05-08T15:53:52.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcae000a150 con 0x7fcae0002860 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fcaf4003fd0 con 0x7fcae0002860 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.7:6789/0 conn(0x7fcae0002860 legacy=0x7fcae0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.439+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.7:6789/0 conn(0x7fcae0002860 legacy=0x7fcae0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcb085bd640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fcae0009e40 con 0x7fcae0009100 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1155213925 0 0) 0x7fcae0009e40 con 0x7fcae0009100 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 10 monclient(hunting): my global_id is 95715 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcae00065c0 con 0x7fcae0009100 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 413046887 0 0) 0x7fcae00065c0 con 0x7fcae0009100 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fcae0009e40 con 0x7fcae0009100 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fcaf40048d0 con 0x7fcae0009100 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 10 monclient: dump: 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.472 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1106928195 0 0) 0x7fcae0009e40 con 0x7fcae0009100 2024-05-08T15:53:52.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.446+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcae000c3d0 con 0x7fcae0009100 2024-05-08T15:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7852 (unknown 0 0 2758083996) 0x7fcaf4004b80 con 0x7fcae0009100 2024-05-08T15:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcafeffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcafeffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.7:6789/0 conn(0x7fcae0009100 legacy=0x7fcae0009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcafeffd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.7:6789/0 conn(0x7fcae0009100 legacy=0x7fcae0009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.474 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:52.474 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 282997, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:52.475 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.476 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.477 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:52.478 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:52.479 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.480 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:52.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.483 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.484 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.485 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.486 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.487 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.489 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.491 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.493 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.495 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:52.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.497 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.498 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.498 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:52.498 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.81:6812/847627118 conn(0x7fcaec03a410 legacy=0x7fcaec03c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 10 monclient: shutdown 2024-05-08T15:53:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 >> v1:172.21.15.7:6791/0 conn(0x7fcb000a41b0 legacy=0x7fcb001ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb06b5a640 1 -- 172.21.15.7:0/1035837860 reap_dead start 2024-05-08T15:53:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb06b5a640 1 -- 172.21.15.7:0/1035837860 reap_dead start 2024-05-08T15:53:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 shutdown_connections 2024-05-08T15:53:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 >> 172.21.15.7:0/1035837860 conn(0x7fcb000a9fa0 msgr2=0x7fcb001307d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 shutdown_connections 2024-05-08T15:53:52.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.447+0000 7fcb085bd640 1 -- 172.21.15.7:0/1035837860 wait complete. 2024-05-08T15:53:52.500 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:52.500 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon b 2024-05-08T15:53:52.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.608+0000 7f91a22df640 10 monclient: get_monmap_and_config 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.608+0000 7f91a22df640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.608+0000 7f91a22df640 10 monclient: monmap: 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:52.609797+0000 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:52.609797+0000 2024-05-08T15:53:52.615 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.609+0000 7f91a22df640 1 Processor -- start 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.609+0000 7f91a22df640 1 -- start start 2024-05-08T15:53:52.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.609+0000 7f91a22df640 10 monclient: init 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 10 monclient: _add_conns ranks=[7,6,3,1,0,8,2,4,5] 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 10 monclient(hunting): picked mon.noname-i con 0x7f919c137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 10 monclient(hunting): picked mon.noname-d con 0x7f919c0a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 10 monclient(hunting): picked mon.noname-g con 0x7f919c0a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c0a2c70 con 0x7f919c0a41b0 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c0a2f50 con 0x7f919c0a49a0 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c0a1670 con 0x7f919c137940 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f91a22df640 10 monclient(hunting): authenticate will time out at 1240.724872s 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f9193fff640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f919c0a49a0 0x7f919c13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:42966/0 (socket says 172.21.15.7:42966) 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f9193fff640 1 -- 172.21.15.7:0/4047468372 learned_addr learned my addr 172.21.15.7:0/4047468372 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2627967031 0 0) 0x7f919c0a2f50 con 0x7f919c0a49a0 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): my global_id is 124350 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9188003520 con 0x7f919c0a49a0 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 602798990 0 0) 0x7f919c0a1670 con 0x7f919c137940 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): my global_id is 115000 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f919c0a2f50 con 0x7f919c137940 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3018652942 0 0) 0x7f919c0a2c70 con 0x7f919c0a41b0 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient(hunting): my global_id is 76638 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f919c0a1670 con 0x7f919c0a41b0 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2853495282 0 0) 0x7f9188003520 con 0x7f919c0a49a0 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 >> v1:172.21.15.81:6792/0 conn(0x7f919c137940 legacy=0x7f919c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 >> v1:172.21.15.81:6790/0 conn(0x7f919c0a41b0 legacy=0x7f919c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 monclient: found mon.noname-d 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c0a39f0 con 0x7f919c0a49a0 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f91840017b0 con 0x7f919c0a49a0 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.610+0000 7f919b7fe640 10 monclient: dump: 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.620 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 5 monclient: authenticate success, global_id 124350 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9184001ad0 con 0x7f919c0a49a0 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f919b7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f919b7fe640 1 -- 172.21.15.7:0/4047468372 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9184004be0 con 0x7f919c0a49a0 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f919b7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 4 set_mon_vals no callback set 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f919b7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f919b7fe640 10 monclient: dump: 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.622 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 10 monclient: shutdown 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 1 -- 172.21.15.7:0/4047468372 >> v1:172.21.15.7:6792/0 conn(0x7f919c0a49a0 legacy=0x7f919c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 1 -- 172.21.15.7:0/4047468372 shutdown_connections 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 1 -- 172.21.15.7:0/4047468372 >> 172.21.15.7:0/4047468372 conn(0x7f919c0a9fa0 msgr2=0x7f919c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 1 -- 172.21.15.7:0/4047468372 shutdown_connections 2024-05-08T15:53:52.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.611+0000 7f91a22df640 1 -- 172.21.15.7:0/4047468372 wait complete. 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.612+0000 7f91a22df640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.612+0000 7f91a22df640 10 monclient: monmap: 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.624 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.612+0000 7f91a22df640 1 Processor -- start 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.612+0000 7f91a22df640 1 -- start start 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.612+0000 7f91a22df640 10 monclient: init 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 10 monclient: _add_conns ranks=[7,1,5,4,0,2,3,8,6] 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 10 monclient(hunting): picked mon.noname-i con 0x7f919c0a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 10 monclient(hunting): picked mon.noname-f con 0x7f919c0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 10 monclient(hunting): picked mon.noname-h con 0x7f919c137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c0a1f30 con 0x7f919c0a49a0 2024-05-08T15:53:52.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c135a40 con 0x7f919c137940 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f919c0a0f60 con 0x7f919c0a41b0 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91a22df640 10 monclient(hunting): authenticate will time out at 1240.727739s 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f919bfff640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7f919c0a41b0 0x7f919c131e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:38134/0 (socket says 172.21.15.7:38134) 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f919bfff640 1 -- 172.21.15.7:0/69687213 learned_addr learned my addr 172.21.15.7:0/69687213 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3622844338 0 0) 0x7f919c135a40 con 0x7f919c137940 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): my global_id is 6485 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f917c003520 con 0x7f919c137940 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3078430122 0 0) 0x7f919c0a0f60 con 0x7f919c0a41b0 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): my global_id is 115009 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f919c135a40 con 0x7f919c0a41b0 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1952039357 0 0) 0x7f919c0a39f0 con 0x7f919c0a49a0 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): my global_id is 6265 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f919c0a0f60 con 0x7f919c0a49a0 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 887332947 0 0) 0x7f919c135a40 con 0x7f919c0a41b0 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f919c0a39f0 con 0x7f919c0a41b0 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2058539828 0 0) 0x7f917c003520 con 0x7f919c137940 2024-05-08T15:53:52.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f919c135a40 con 0x7f919c137940 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f918c001910 con 0x7f919c0a41b0 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.613+0000 7f91997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): got monmap 1 from mon.noname-i (according to old e1) 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): dump: 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.628 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9194001dd0 con 0x7f919c137940 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): dump: 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.630 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4236372855 0 0) 0x7f919c0a0f60 con 0x7f919c0a49a0 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f917c003520 con 0x7f919c0a49a0 2024-05-08T15:53:52.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9184004b20 con 0x7f919c0a49a0 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient(hunting): dump: 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.632 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1213481951 0 0) 0x7f919c0a39f0 con 0x7f919c0a41b0 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.81:6791/0 conn(0x7f919c137940 legacy=0x7f919c1d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.81:6789/0 conn(0x7f919c0a49a0 legacy=0x7f919c132550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 monclient: found mon.i 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c132c60 con 0x7f919c0a41b0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 5 monclient: authenticate success, global_id 115009 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 10 monclient: _renew_subs 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f919c135760 con 0x7f919c0a41b0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 10 monclient: _renew_subs 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 10 monclient: _send_mon_message to mon.i at v1:172.21.15.81:6792/0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.81:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f919c134590 con 0x7f919c0a41b0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f918c001cf0 con 0x7f919c0a41b0 2024-05-08T15:53:52.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f918c003c00 con 0x7f919c0a41b0 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f919affd640 4 set_mon_vals no callback set 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91997fa640 10 monclient: dump: 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.635 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.614+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f91800065c0 con 0x7f9180002860 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399645527 0 0) 0x7f91800065c0 con 0x7f9180002860 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 10 monclient(hunting): my global_id is 95843 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f919c0a39f0 con 0x7f9180002860 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f918c003e20 con 0x7f919c0a41b0 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.7 v1:172.21.15.81:6792/0 8 ==== osd_map(300..300 src has 1..300) ==== 3898+0+0 (unknown 3456018804 0 0) 0x7f918c046e10 con 0x7f919c0a41b0 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91a087c640 10 monclient: get_auth_request con 0x7f917c03a240 auth_method 0 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3253173655 0 0) 0x7f919c0a39f0 con 0x7f9180002860 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f91800065c0 con 0x7f9180002860 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f91840010a0 con 0x7f9180002860 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 10 monclient: dump: 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.637 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.638 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 209630465 0 0) 0x7f91800065c0 con 0x7f9180002860 2024-05-08T15:53:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.615+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f918000a150 con 0x7f9180002860 2024-05-08T15:53:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.616+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f9184007480 con 0x7f9180002860 2024-05-08T15:53:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.616+0000 7f91997fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.616+0000 7f91997fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.616+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.7:6790/0 conn(0x7f9180002860 legacy=0x7f9180004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.616+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.7:6790/0 conn(0x7f9180002860 legacy=0x7f9180004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91a22df640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f9180009e20 con 0x7f91800090e0 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2631581830 0 0) 0x7f9180009e20 con 0x7f91800090e0 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91997fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91997fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91997fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91997fa640 10 monclient(hunting): my global_id is 95852 2024-05-08T15:53:52.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.622+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91800065c0 con 0x7f91800090e0 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2589283226 0 0) 0x7f91800065c0 con 0x7f91800090e0 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9180009e20 con 0x7f91800090e0 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9184005680 con 0x7f91800090e0 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 10 monclient: dump: 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.642 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1231492407 0 0) 0x7f9180009e20 con 0x7f91800090e0 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f918000c390 con 0x7f91800090e0 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7682 (unknown 0 0 3410545271) 0x7f9184007630 con 0x7f91800090e0 2024-05-08T15:53:52.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.7:6790/0 conn(0x7f91800090e0 legacy=0x7f9180009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91997fa640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.7:6790/0 conn(0x7f91800090e0 legacy=0x7f9180009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:52.644 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 283174, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:52.645 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.646 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.647 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.648 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:52.649 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.650 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.651 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.652 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:52.653 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.654 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.655 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.656 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.657 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.658 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.659 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.660 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.661 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.663 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:52.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.666 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:52.666 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.81:6812/847627118 conn(0x7f917c03a240 legacy=0x7f917c03c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 10 monclient: shutdown 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 >> v1:172.21.15.81:6792/0 conn(0x7f919c0a41b0 legacy=0x7f919c131e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a087c640 1 -- 172.21.15.7:0/69687213 reap_dead start 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a087c640 1 -- 172.21.15.7:0/69687213 reap_dead start 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 shutdown_connections 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 >> 172.21.15.7:0/69687213 conn(0x7f919c0a9fa0 msgr2=0x7f919c1308d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 shutdown_connections 2024-05-08T15:53:52.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.623+0000 7f91a22df640 1 -- 172.21.15.7:0/69687213 wait complete. 2024-05-08T15:53:52.668 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:52.668 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon c 2024-05-08T15:53:52.668 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:53:52.768 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38981]: starting. 2024-05-08T15:53:52.768 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38981]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38977 2024-05-08T15:53:52.769 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[38981]: shutting down. 2024-05-08T15:53:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.787+0000 7fe5fcb8f640 10 monclient: get_monmap_and_config 2024-05-08T15:53:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.787+0000 7fe5fcb8f640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.787+0000 7fe5fcb8f640 10 monclient: monmap: 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:52.788679+0000 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:52.788679+0000 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.784 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.785 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.785 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.788+0000 7fe5fcb8f640 1 Processor -- start 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.788+0000 7fe5fcb8f640 1 -- start start 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.788+0000 7fe5fcb8f640 10 monclient: init 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 10 monclient: _add_conns ranks=[7,6,2,8,4,3,1,0,5] 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 10 monclient(hunting): picked mon.noname-i con 0x7fe5f80a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 10 monclient(hunting): picked mon.noname-d con 0x7fe5f80a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 10 monclient(hunting): picked mon.noname-b con 0x7fe5f813e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f80a1660 con 0x7fe5f813e310 2024-05-08T15:53:52.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f80a28f0 con 0x7fe5f80a5780 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f80a2bd0 con 0x7fe5f80a53a0 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 10 monclient(hunting): authenticate will time out at 1240.903916s 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f5d9b640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fe5f80a5780 0x7fe5f813cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:42974/0 (socket says 172.21.15.7:42974) 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f5d9b640 1 -- 172.21.15.7:0/3508043342 learned_addr learned my addr 172.21.15.7:0/3508043342 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f6d9d640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fe5f813e310 0x7fe5f8140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43644/0 (socket says 172.21.15.7:43644) 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4155906824 0 0) 0x7fe5f80a28f0 con 0x7fe5f80a5780 2024-05-08T15:53:52.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): my global_id is 124368 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5e0003520 con 0x7fe5f80a5780 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576725579 0 0) 0x7fe5f80a1660 con 0x7fe5f813e310 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): my global_id is 95861 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5f80a28f0 con 0x7fe5f813e310 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3260821643 0 0) 0x7fe5f80a2bd0 con 0x7fe5f80a53a0 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient(hunting): my global_id is 115027 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5f80a1660 con 0x7fe5f80a53a0 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 813681825 0 0) 0x7fe5e0003520 con 0x7fe5f80a5780 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 >> v1:172.21.15.81:6792/0 conn(0x7fe5f80a53a0 legacy=0x7fe5f80a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 >> v1:172.21.15.7:6790/0 conn(0x7fe5f813e310 legacy=0x7fe5f8140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 monclient: found mon.noname-d 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:53:52.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f80a6730 con 0x7fe5f80a5780 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 5 monclient: authenticate success, global_id 124368 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe5ec0017b0 con 0x7fe5f80a5780 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5f559a640 10 monclient: dump: 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.799 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.789+0000 7fe5fcb8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe5ec001ad0 con 0x7fe5f80a5780 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5f559a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5f559a640 1 -- 172.21.15.7:0/3508043342 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe5ec004be0 con 0x7fe5f80a5780 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5f559a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5f559a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5f559a640 10 monclient: dump: 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.801 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 4 set_mon_vals no callback set 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 10 monclient: shutdown 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/3508043342 >> v1:172.21.15.7:6792/0 conn(0x7fe5f80a5780 legacy=0x7fe5f813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/3508043342 shutdown_connections 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/3508043342 >> 172.21.15.7:0/3508043342 conn(0x7fe5f80a9fa0 msgr2=0x7fe5f80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/3508043342 shutdown_connections 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.790+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/3508043342 wait complete. 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.791+0000 7fe5fcb8f640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.791+0000 7fe5fcb8f640 10 monclient: monmap: 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.803 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 1 Processor -- start 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 1 -- start start 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient: init 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient: _add_conns ranks=[6,5,0,3,1,2,8,4,7] 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient(hunting): picked mon.noname-d con 0x7fe5f80a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:52.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient(hunting): picked mon.noname-h con 0x7fe5f80a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient(hunting): picked mon.noname-a con 0x7fe5f813e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f8136790 con 0x7fe5f813e310 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f8137f20 con 0x7fe5f80a5780 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5f80a43c0 con 0x7fe5f80a53a0 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5fcb8f640 10 monclient(hunting): authenticate will time out at 1240.907189s 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5f659c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fe5f80a53a0 0x7fe5f81ccfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:42990/0 (socket says 172.21.15.7:42990) 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5f659c640 1 -- 172.21.15.7:0/418416784 learned_addr learned my addr 172.21.15.7:0/418416784 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3561293725 0 0) 0x7fe5f80a43c0 con 0x7fe5f80a53a0 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): my global_id is 124377 2024-05-08T15:53:52.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5d8003520 con 0x7fe5f80a53a0 2024-05-08T15:53:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3643751666 0 0) 0x7fe5f8136790 con 0x7fe5f813e310 2024-05-08T15:53:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 10 monclient(hunting): my global_id is 95724 2024-05-08T15:53:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.792+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5f80a43c0 con 0x7fe5f813e310 2024-05-08T15:53:52.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 231604211 0 0) 0x7fe5d8003520 con 0x7fe5f80a53a0 2024-05-08T15:53:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5f8136790 con 0x7fe5f80a53a0 2024-05-08T15:53:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe5e8001b30 con 0x7fe5f80a53a0 2024-05-08T15:53:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:52.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.811 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1433366264 0 0) 0x7fe5f8137f20 con 0x7fe5f80a5780 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): my global_id is 6494 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5d8003520 con 0x7fe5f80a5780 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 640977861 0 0) 0x7fe5f80a43c0 con 0x7fe5f813e310 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5f8137f20 con 0x7fe5f813e310 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe5f0001b70 con 0x7fe5f813e310 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:52.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.813 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3494299366 0 0) 0x7fe5f8136790 con 0x7fe5f80a53a0 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.81:6791/0 conn(0x7fe5f80a5780 legacy=0x7fe5f81d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.7:6789/0 conn(0x7fe5f813e310 legacy=0x7fe5f81d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 monclient: found mon.d 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5f80a1e00 con 0x7fe5f80a53a0 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5fcb8f640 5 monclient: authenticate success, global_id 124377 2024-05-08T15:53:52.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5fcb8f640 10 monclient: _renew_subs 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5fcb8f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe5f8137c40 con 0x7fe5f80a53a0 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5fcb8f640 10 monclient: _renew_subs 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5fcb8f640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fe5f8136370 con 0x7fe5f80a53a0 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe5e80038c0 con 0x7fe5f80a53a0 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe5e8003e30 con 0x7fe5f80a53a0 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5f4d99640 4 set_mon_vals no callback set 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.793+0000 7fe5e6ffd640 10 monclient: dump: 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.815 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.794+0000 7fe5fcb8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.794+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe5dc0065c0 con 0x7fe5dc002860 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.794+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fe5e8004050 con 0x7fe5f80a53a0 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(301..301 src has 1..301) ==== 3506+0+0 (unknown 1436118537 0 0) 0x7fe5e8046db0 con 0x7fe5f80a53a0 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1798322991 0 0) 0x7fe5dc0065c0 con 0x7fe5dc002860 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient(hunting): my global_id is 124294 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5f8136790 con 0x7fe5dc002860 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5f6d9d640 10 monclient: get_auth_request con 0x7fe5d803a120 auth_method 0 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4223738971 0 0) 0x7fe5f8136790 con 0x7fe5dc002860 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5dc0065c0 con 0x7fe5dc002860 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe5ec001430 con 0x7fe5dc002860 2024-05-08T15:53:52.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient: dump: 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.818 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1929113804 0 0) 0x7fe5dc0065c0 con 0x7fe5dc002860 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5dc00a150 con 0x7fe5dc002860 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fe5ec0017b0 con 0x7fe5dc002860 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.7:6791/0 conn(0x7fe5dc002860 legacy=0x7fe5dc004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.795+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.7:6791/0 conn(0x7fe5dc002860 legacy=0x7fe5dc004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5fcb8f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fe5dc009e40 con 0x7fe5dc009100 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 714990855 0 0) 0x7fe5dc009e40 con 0x7fe5dc009100 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 10 monclient(hunting): my global_id is 124303 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5dc0065c0 con 0x7fe5dc009100 2024-05-08T15:53:52.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3112710601 0 0) 0x7fe5dc0065c0 con 0x7fe5dc009100 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe5dc009e40 con 0x7fe5dc009100 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe5ec004ad0 con 0x7fe5dc009100 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.802+0000 7fe5e6ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 10 monclient: dump: 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.822 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 937601264 0 0) 0x7fe5dc009e40 con 0x7fe5dc009100 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe5dc00c3d0 con 0x7fe5dc009100 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7511 (unknown 0 0 2588644485) 0x7fe5ec0053b0 con 0x7fe5dc009100 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.7:6791/0 conn(0x7fe5dc009100 legacy=0x7fe5dc009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5e6ffd640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.7:6791/0 conn(0x7fe5dc009100 legacy=0x7fe5dc009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 40974, 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:52.824 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.825 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.826 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:52.827 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.828 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:52.829 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:52.830 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.831 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.832 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.833 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.834 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.835 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.836 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:52.837 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.838 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.839 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.840 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.841 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:52.842 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.81:6812/847627118 conn(0x7fe5d803a120 legacy=0x7fe5d803c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 10 monclient: shutdown 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 >> v1:172.21.15.7:6792/0 conn(0x7fe5f80a53a0 legacy=0x7fe5f81ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5f6d9d640 1 -- 172.21.15.7:0/418416784 reap_dead start 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5f6d9d640 1 -- 172.21.15.7:0/418416784 reap_dead start 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 shutdown_connections 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 >> 172.21.15.7:0/418416784 conn(0x7fe5f80a9fa0 msgr2=0x7fe5f8140c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 shutdown_connections 2024-05-08T15:53:52.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.803+0000 7fe5fcb8f640 1 -- 172.21.15.7:0/418416784 wait complete. 2024-05-08T15:53:52.845 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:52.845 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon d 2024-05-08T15:53:52.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:53:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.952+0000 7f2bb7d3b640 10 monclient: get_monmap_and_config 2024-05-08T15:53:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.952+0000 7f2bb7d3b640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.952+0000 7f2bb7d3b640 10 monclient: monmap: 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:52.953935+0000 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:52.953935+0000 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.950 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.953+0000 7f2bb7d3b640 1 Processor -- start 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.953+0000 7f2bb7d3b640 1 -- start start 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.953+0000 7f2bb7d3b640 10 monclient: init 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 10 monclient: _add_conns ranks=[5,7,6,4,1,8,2,0,3] 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 10 monclient(hunting): picked mon.noname-h con 0x7f2bb0137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 10 monclient(hunting): picked mon.noname-i con 0x7f2bb00a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 10 monclient(hunting): picked mon.noname-d con 0x7f2bb00a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb00a2c70 con 0x7f2bb0137940 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb00a2f50 con 0x7f2bb00a41b0 2024-05-08T15:53:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb00a1670 con 0x7f2bb00a49a0 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 10 monclient(hunting): authenticate will time out at 1241.068907s 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb62d8640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f2bb00a41b0 0x7f2bb0140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:43004/0 (socket says 172.21.15.7:43004) 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb62d8640 1 -- 172.21.15.7:0/330398151 learned_addr learned my addr 172.21.15.7:0/330398151 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2555267009 0 0) 0x7f2bb00a2f50 con 0x7f2bb00a41b0 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient(hunting): my global_id is 124395 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b9c003520 con 0x7f2bb00a41b0 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3783421686 0 0) 0x7f2b9c003520 con 0x7f2bb00a41b0 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 >> v1:172.21.15.81:6792/0 conn(0x7f2bb00a49a0 legacy=0x7f2bb013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 >> v1:172.21.15.81:6791/0 conn(0x7f2bb0137940 legacy=0x7f2bb0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 monclient: found mon.noname-d 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb00a39f0 con 0x7f2bb00a41b0 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2ba8001b30 con 0x7f2bb00a41b0 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb4ad5640 10 monclient: dump: 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 5 monclient: authenticate success, global_id 124395 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.954+0000 7f2bb7d3b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2ba8001e90 con 0x7f2bb00a41b0 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb4ad5640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 4 set_mon_vals no callback set 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb4ad5640 1 -- 172.21.15.7:0/330398151 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2ba80020c0 con 0x7f2bb00a41b0 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb4ad5640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb4ad5640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb4ad5640 10 monclient: dump: 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 10 monclient: shutdown 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/330398151 >> v1:172.21.15.7:6792/0 conn(0x7f2bb00a41b0 legacy=0x7f2bb0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/330398151 shutdown_connections 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/330398151 >> 172.21.15.7:0/330398151 conn(0x7f2bb00a9fa0 msgr2=0x7f2bb00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/330398151 shutdown_connections 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.955+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/330398151 wait complete. 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 10 monclient: build_initial_monmap 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 10 monclient: monmap: 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 1 Processor -- start 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 1 -- start start 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 10 monclient: init 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 10 monclient: _add_conns ranks=[7,2,3,4,8,5,0,1,6] 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 10 monclient(hunting): picked mon.noname-i con 0x7f2bb00a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.956+0000 7f2bb7d3b640 10 monclient(hunting): picked mon.noname-b con 0x7f2bb00a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 10 monclient(hunting): picked mon.noname-g con 0x7f2bb0137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb01340e0 con 0x7f2bb00a49a0 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb00a1f30 con 0x7f2bb0137940 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb01339b0 con 0x7f2bb00a41b0 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 10 monclient(hunting): authenticate will time out at 1241.071649s 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb52d6640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f2bb00a49a0 0x7f2bb01d0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43658/0 (socket says 172.21.15.7:43658) 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb52d6640 1 -- 172.21.15.7:0/1053389161 learned_addr learned my addr 172.21.15.7:0/1053389161 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3095015863 0 0) 0x7f2bb01340e0 con 0x7f2bb00a49a0 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): my global_id is 95906 2024-05-08T15:53:52.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b94003520 con 0x7f2bb00a49a0 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3585418570 0 0) 0x7f2bb01339b0 con 0x7f2bb00a41b0 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): my global_id is 115045 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bb01340e0 con 0x7f2bb00a41b0 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1818447744 0 0) 0x7f2bb00a39f0 con 0x7f2bb0137940 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): my global_id is 76647 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bb01339b0 con 0x7f2bb0137940 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3142368991 0 0) 0x7f2b94003520 con 0x7f2bb00a49a0 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2bb00a39f0 con 0x7f2bb00a49a0 2024-05-08T15:53:52.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2ba0001a90 con 0x7f2bb00a49a0 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): dump: 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2759602825 0 0) 0x7f2bb01340e0 con 0x7f2bb00a41b0 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2b94003520 con 0x7f2bb00a41b0 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3388767447 0 0) 0x7f2bb01339b0 con 0x7f2bb0137940 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2bb01340e0 con 0x7f2bb0137940 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bac001820 con 0x7f2bb00a41b0 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): dump: 2024-05-08T15:53:52.969 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.970 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2ba8001af0 con 0x7f2bb0137940 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient(hunting): dump: 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.971 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2518982621 0 0) 0x7f2bb00a39f0 con 0x7f2bb00a49a0 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.81:6792/0 conn(0x7f2bb00a41b0 legacy=0x7f2bb0136110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.81:6790/0 conn(0x7f2bb0137940 legacy=0x7f2bb01d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: _finish_hunting 0 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 monclient: found mon.b 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb00a0f60 con 0x7f2bb00a49a0 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: _finish_auth 0 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: _check_auth_tickets 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 5 monclient: authenticate success, global_id 95906 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 10 monclient: _renew_subs 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb0135a10 con 0x7f2bb00a49a0 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 10 monclient: _renew_subs 2024-05-08T15:53:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f2bb01d5a90 con 0x7f2bb00a49a0 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2ba00038c0 con 0x7f2bb00a49a0 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2ba0003db0 con 0x7f2bb00a49a0 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba67fc640 10 monclient: dump: 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.957+0000 7f2ba7fff640 4 set_mon_vals no callback set 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.958+0000 7f2bb7d3b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.958+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2ba0003f50 con 0x7f2bb00a49a0 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.958+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2bb0072ed0 con 0x7f2bb009ba50 2024-05-08T15:53:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(301..301 src has 1..301) ==== 3506+0+0 (unknown 1436118537 0 0) 0x7f2ba0046db0 con 0x7f2bb00a49a0 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2239988786 0 0) 0x7f2b94003520 con 0x7f2bb009ba50 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient(hunting): my global_id is 124404 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2bb00a39f0 con 0x7f2bb009ba50 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2bb62d8640 10 monclient: get_auth_request con 0x7f2b9403a580 auth_method 0 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 758968674 0 0) 0x7f2bb00a39f0 con 0x7f2bb009ba50 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2b94003520 con 0x7f2bb009ba50 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bac001c70 con 0x7f2bb009ba50 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient: dump: 2024-05-08T15:53:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.977 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1307977937 0 0) 0x7f2b94003520 con 0x7f2bb009ba50 2024-05-08T15:53:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2bb0136930 con 0x7f2bb009ba50 2024-05-08T15:53:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2bac005ab0 con 0x7f2bb009ba50 2024-05-08T15:53:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.7:6792/0 conn(0x7f2bb009ba50 legacy=0x7f2bb009be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.959+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.7:6792/0 conn(0x7f2bb009ba50 legacy=0x7f2bb009be30 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2bb7d3b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2bb0141cd0 con 0x7f2bb009c540 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2344115476 0 0) 0x7f2bac003b00 con 0x7f2bb009c540 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 10 monclient(hunting): _negotiate 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 10 monclient(hunting): my global_id is 124413 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b94003520 con 0x7f2bb009c540 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3989420056 0 0) 0x7f2b94003520 con 0x7f2bb009c540 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2bac003b00 con 0x7f2bb009c540 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2bac004300 con 0x7f2bb009c540 2024-05-08T15:53:52.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.966+0000 7f2ba67fc640 10 monclient: dump: 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3302088255 0 0) 0x7f2bac003b00 con 0x7f2bb009c540 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2bb00753d0 con 0x7f2bb009c540 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7511 (unknown 0 0 670727541) 0x7f2bac005790 con 0x7f2bb009c540 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2ba67fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2ba67fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.7:6792/0 conn(0x7f2bb009c540 legacy=0x7f2bb01d9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2ba67fc640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.7:6792/0 conn(0x7f2bb009c540 legacy=0x7f2bb01d9c80 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 41138, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:52.983 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.984 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.985 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.986 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:52.987 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.988 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.989 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.990 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.991 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.992 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.994 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.995 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.996 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.997 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:52.998 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:52.999 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.000 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.001 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:53.001 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.81:6812/847627118 conn(0x7f2b9403a580 legacy=0x7f2b9403ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 10 monclient: shutdown 2024-05-08T15:53:53.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 >> v1:172.21.15.7:6790/0 conn(0x7f2bb00a49a0 legacy=0x7f2bb01d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb62d8640 1 -- 172.21.15.7:0/1053389161 reap_dead start 2024-05-08T15:53:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb62d8640 1 -- 172.21.15.7:0/1053389161 reap_dead start 2024-05-08T15:53:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 shutdown_connections 2024-05-08T15:53:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 >> 172.21.15.7:0/1053389161 conn(0x7f2bb00a9fa0 msgr2=0x7f2bb01308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 shutdown_connections 2024-05-08T15:53:53.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:52.967+0000 7f2bb7d3b640 1 -- 172.21.15.7:0/1053389161 wait complete. 2024-05-08T15:53:53.003 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:53.003 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon f 2024-05-08T15:53:53.004 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.113+0000 7f77daa3e640 10 monclient: get_monmap_and_config 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.113+0000 7f77daa3e640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.113+0000 7f77daa3e640 10 monclient: monmap: 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:53.114419+0000 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:53.114419+0000 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.109 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.110 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 1 Processor -- start 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 1 -- start start 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient: init 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient: _add_conns ranks=[7,4,2,5,1,6,3,8,0] 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient(hunting): picked mon.noname-i con 0x7f77d4137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient(hunting): picked mon.noname-c con 0x7f77d40a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient(hunting): picked mon.noname-b con 0x7f77d40a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d40a2c70 con 0x7f77d40a41b0 2024-05-08T15:53:53.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d40a2f50 con 0x7f77d40a49a0 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d40a1670 con 0x7f77d4137940 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77daa3e640 10 monclient(hunting): authenticate will time out at 1241.229359s 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77d8fdb640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f77d40a41b0 0x7f77d4140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43668/0 (socket says 172.21.15.7:43668) 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77d8fdb640 1 -- 172.21.15.7:0/1763150563 learned_addr learned my addr 172.21.15.7:0/1763150563 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.114+0000 7f77d37fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f77d40a49a0 0x7f77d413cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:52348/0 (socket says 172.21.15.7:52348) 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001523852 0 0) 0x7f77d40a2c70 con 0x7f77d40a41b0 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): my global_id is 95915 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77bc003520 con 0x7f77d40a41b0 2024-05-08T15:53:53.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2205923206 0 0) 0x7f77d40a2f50 con 0x7f77d40a49a0 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): my global_id is 124312 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77d40a2c70 con 0x7f77d40a49a0 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1504571907 0 0) 0x7f77d40a1670 con 0x7f77d4137940 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient(hunting): my global_id is 115054 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77d40a2f50 con 0x7f77d4137940 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2191809545 0 0) 0x7f77d40a2c70 con 0x7f77d40a49a0 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 >> v1:172.21.15.81:6792/0 conn(0x7f77d4137940 legacy=0x7f77d4137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 >> v1:172.21.15.7:6790/0 conn(0x7f77d40a41b0 legacy=0x7f77d4140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 monclient: found mon.noname-c 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d40a39f0 con 0x7f77d40a49a0 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77c8001790 con 0x7f77d40a49a0 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: dump: 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.122 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.123 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77daa3e640 5 monclient: authenticate success, global_id 124312 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77daa3e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f77c8001a70 con 0x7f77d40a49a0 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 1 -- 172.21.15.7:0/1763150563 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77c8001ce0 con 0x7f77d40a49a0 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77daa3e640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77daa3e640 4 set_mon_vals no callback set 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.115+0000 7f77d2ffd640 10 monclient: dump: 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.124 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 10 monclient: shutdown 2024-05-08T15:53:53.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 1 -- 172.21.15.7:0/1763150563 >> v1:172.21.15.7:6791/0 conn(0x7f77d40a49a0 legacy=0x7f77d413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 1 -- 172.21.15.7:0/1763150563 shutdown_connections 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 1 -- 172.21.15.7:0/1763150563 >> 172.21.15.7:0/1763150563 conn(0x7f77d40a9fa0 msgr2=0x7f77d40aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 1 -- 172.21.15.7:0/1763150563 shutdown_connections 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 1 -- 172.21.15.7:0/1763150563 wait complete. 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.116+0000 7f77daa3e640 10 monclient: monmap: 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:53.126 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 1 Processor -- start 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 1 -- start start 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient: init 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient: _add_conns ranks=[1,4,3,8,6,5,0,7,2] 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient(hunting): picked mon.noname-f con 0x7f77d40a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient(hunting): picked mon.noname-c con 0x7f77d40a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient(hunting): picked mon.noname-g con 0x7f77d4137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d40a0f60 con 0x7f77d40a41b0 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d4133710 con 0x7f77d4137940 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d4137640 con 0x7f77d40a49a0 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77daa3e640 10 monclient(hunting): authenticate will time out at 1241.232336s 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77d37fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f77d40a49a0 0x7f77d4136f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:52358/0 (socket says 172.21.15.7:52358) 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77d37fe640 1 -- 172.21.15.7:0/1932497772 learned_addr learned my addr 172.21.15.7:0/1932497772 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.117+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3368735431 0 0) 0x7f77d4137640 con 0x7f77d40a49a0 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): my global_id is 124321 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77c0003520 con 0x7f77d40a49a0 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278549062 0 0) 0x7f77d4133710 con 0x7f77d4137940 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): my global_id is 76656 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77d4137640 con 0x7f77d4137940 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570687348 0 0) 0x7f77d40a0f60 con 0x7f77d40a41b0 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): my global_id is 6301 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77d4133710 con 0x7f77d40a41b0 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3711597098 0 0) 0x7f77c0003520 con 0x7f77d40a49a0 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77d40a0f60 con 0x7f77d40a49a0 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77c8001750 con 0x7f77d40a49a0 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): dump: 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.131 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.132 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.133 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3450485625 0 0) 0x7f77d4137640 con 0x7f77d4137940 2024-05-08T15:53:53.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77c0003520 con 0x7f77d4137940 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77cc0017d0 con 0x7f77d4137940 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): dump: 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.134 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2511343896 0 0) 0x7f77d4133710 con 0x7f77d40a41b0 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77d4137640 con 0x7f77d40a41b0 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77c40019b0 con 0x7f77d40a41b0 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient(hunting): dump: 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.135 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.136 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1816577040 0 0) 0x7f77d40a0f60 con 0x7f77d40a49a0 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.81:6790/0 conn(0x7f77d4137940 legacy=0x7f77d41d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.81:6789/0 conn(0x7f77d40a41b0 legacy=0x7f77d41cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 monclient: found mon.c 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d4134390 con 0x7f77d40a49a0 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77daa3e640 5 monclient: authenticate success, global_id 124321 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77daa3e640 10 monclient: _renew_subs 2024-05-08T15:53:53.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77daa3e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f77d41345c0 con 0x7f77d40a49a0 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77daa3e640 10 monclient: _renew_subs 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77daa3e640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f77d41d5f00 con 0x7f77d40a49a0 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f77c8001a70 con 0x7f77d40a49a0 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77c8005230 con 0x7f77d40a49a0 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d1ffb640 4 set_mon_vals no callback set 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.118+0000 7f77d0ff9640 10 monclient: dump: 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.138 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f77c80105b0 con 0x7f77d40a49a0 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77daa3e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f77b40065c0 con 0x7f77b4002860 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(301..301 src has 1..301) ==== 3506+0+0 (unknown 1436118537 0 0) 0x7f77c8046c70 con 0x7f77d40a49a0 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d8fdb640 10 monclient: get_auth_request con 0x7f77c003a3f0 auth_method 0 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483405794 0 0) 0x7f77b40065c0 con 0x7f77b4002860 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 10 monclient(hunting): my global_id is 6310 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.119+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77d40a0f60 con 0x7f77b4002860 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 882141670 0 0) 0x7f77d40a0f60 con 0x7f77b4002860 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77b40065c0 con 0x7f77b4002860 2024-05-08T15:53:53.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77c4003930 con 0x7f77b4002860 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 10 monclient: dump: 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.141 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2097178752 0 0) 0x7f77b40065c0 con 0x7f77b4002860 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77b400a150 con 0x7f77b4002860 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f77c4003d00 con 0x7f77b4002860 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.81:6789/0 conn(0x7f77b4002860 legacy=0x7f77b4004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.120+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.81:6789/0 conn(0x7f77b4002860 legacy=0x7f77b4004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77daa3e640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f77b4009e40 con 0x7f77b4009100 2024-05-08T15:53:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1432128421 0 0) 0x7f77b4009e40 con 0x7f77b4009100 2024-05-08T15:53:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 10 monclient(hunting): my global_id is 6319 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77b40065c0 con 0x7f77b4009100 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4256631819 0 0) 0x7f77b40065c0 con 0x7f77b4009100 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f77b4009e40 con 0x7f77b4009100 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f77c4004470 con 0x7f77b4009100 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 10 monclient: dump: 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.145 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2034601296 0 0) 0x7f77b4009e40 con 0x7f77b4009100 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.127+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f77b400c3d0 con 0x7f77b4009100 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7684 (unknown 0 0 2968271050) 0x7f77c40058d0 con 0x7f77b4009100 2024-05-08T15:53:53.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77d0ff9640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77d0ff9640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.81:6789/0 conn(0x7f77b4009100 legacy=0x7f77b4009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77d0ff9640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.81:6789/0 conn(0x7f77b4009100 legacy=0x7f77b4009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 425444, 2024-05-08T15:53:53.147 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.148 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.149 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:53.150 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.151 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:53.152 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:53.153 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:53.154 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.155 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.156 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.158 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.159 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.160 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.161 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.162 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.163 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.164 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.165 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 4 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:53.166 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.81:6812/847627118 conn(0x7f77c003a3f0 legacy=0x7f77c003c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 10 monclient: shutdown 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 >> v1:172.21.15.7:6791/0 conn(0x7f77d40a49a0 legacy=0x7f77d4136f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77d8fdb640 1 -- 172.21.15.7:0/1932497772 reap_dead start 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77d8fdb640 1 -- 172.21.15.7:0/1932497772 reap_dead start 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 shutdown_connections 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 >> 172.21.15.7:0/1932497772 conn(0x7f77d40a9fa0 msgr2=0x7f77d4130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 shutdown_connections 2024-05-08T15:53:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.128+0000 7f77daa3e640 1 -- 172.21.15.7:0/1932497772 wait complete. 2024-05-08T15:53:53.169 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:53.169 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon g 2024-05-08T15:53:53.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:53:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.277+0000 7efc9588c640 10 monclient: get_monmap_and_config 2024-05-08T15:53:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.277+0000 7efc9588c640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.277+0000 7efc9588c640 10 monclient: monmap: 2024-05-08T15:53:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.273 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:53.278730+0000 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:53.278730+0000 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.274 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.275 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 1 Processor -- start 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 1 -- start start 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient: init 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient: _add_conns ranks=[8,2,5,3,4,7,1,6,0] 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient(hunting): picked mon.noname-e con 0x7efc90137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient(hunting): picked mon.noname-b con 0x7efc900a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient(hunting): picked mon.noname-h con 0x7efc900a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc900a2c70 con 0x7efc900a49a0 2024-05-08T15:53:53.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc900a2f50 con 0x7efc900a41b0 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc900a1670 con 0x7efc90137940 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc9588c640 10 monclient(hunting): authenticate will time out at 1241.393526s 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.278+0000 7efc8effd640 1 -- >> v1:172.21.15.7:6793/0 conn(0x7efc90137940 legacy=0x7efc90137d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8e7fc640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7efc900a49a0 0x7efc9013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43676/0 (socket says 172.21.15.7:43676) 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8e7fc640 1 -- 172.21.15.7:0/931282893 learned_addr learned my addr 172.21.15.7:0/931282893 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3355411275 0 0) 0x7efc900a2c70 con 0x7efc900a49a0 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): my global_id is 95924 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc78003520 con 0x7efc900a49a0 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1194323215 0 0) 0x7efc900a2f50 con 0x7efc900a41b0 2024-05-08T15:53:53.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient(hunting): my global_id is 6539 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc900a2c70 con 0x7efc900a41b0 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 836220582 0 0) 0x7efc78003520 con 0x7efc900a49a0 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 >> v1:172.21.15.7:6793/0 conn(0x7efc90137940 legacy=0x7efc90137d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 >> v1:172.21.15.81:6791/0 conn(0x7efc900a41b0 legacy=0x7efc90140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 monclient: found mon.noname-b 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc900a39f0 con 0x7efc900a49a0 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc9588c640 5 monclient: authenticate success, global_id 95924 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc840017b0 con 0x7efc900a49a0 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc9588c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: dump: 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.287 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc9588c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:53.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efc84001ad0 con 0x7efc900a49a0 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 1 -- 172.21.15.7:0/931282893 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc84001d40 con 0x7efc900a49a0 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc9588c640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc9588c640 4 set_mon_vals no callback set 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.279+0000 7efc8dffb640 10 monclient: dump: 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.289 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 10 monclient: shutdown 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 1 -- 172.21.15.7:0/931282893 >> v1:172.21.15.7:6790/0 conn(0x7efc900a49a0 legacy=0x7efc9013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 1 -- 172.21.15.7:0/931282893 shutdown_connections 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 1 -- 172.21.15.7:0/931282893 >> 172.21.15.7:0/931282893 conn(0x7efc900a9fa0 msgr2=0x7efc900aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 1 -- 172.21.15.7:0/931282893 shutdown_connections 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 1 -- 172.21.15.7:0/931282893 wait complete. 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.280+0000 7efc9588c640 10 monclient: monmap: 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.291 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 1 Processor -- start 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 1 -- start start 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient: init 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient: _add_conns ranks=[2,4,0,5,7,1,6,3,8] 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient(hunting): picked mon.noname-b con 0x7efc900a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient(hunting): picked mon.noname-c con 0x7efc900a49a0 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient(hunting): picked mon.noname-a con 0x7efc90137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc901396b0 con 0x7efc90137940 2024-05-08T15:53:53.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc900a1f30 con 0x7efc900a41b0 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc900a0f60 con 0x7efc900a49a0 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc9588c640 10 monclient(hunting): authenticate will time out at 1241.396412s 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc8effd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7efc900a41b0 0x7efc90140f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43686/0 (socket says 172.21.15.7:43686) 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.281+0000 7efc8effd640 1 -- 172.21.15.7:0/2476417351 learned_addr learned my addr 172.21.15.7:0/2476417351 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522823247 0 0) 0x7efc900a1f30 con 0x7efc900a41b0 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): my global_id is 95933 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc7c003520 con 0x7efc900a41b0 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2353589576 0 0) 0x7efc900a0f60 con 0x7efc900a49a0 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): my global_id is 124330 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc900a1f30 con 0x7efc900a49a0 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2939389699 0 0) 0x7efc901396b0 con 0x7efc90137940 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): my global_id is 95733 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc900a0f60 con 0x7efc90137940 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4289193244 0 0) 0x7efc7c003520 con 0x7efc900a41b0 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efc901396b0 con 0x7efc900a41b0 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1125452600 0 0) 0x7efc900a1f30 con 0x7efc900a49a0 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efc7c003520 con 0x7efc900a49a0 2024-05-08T15:53:53.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc800042f0 con 0x7efc900a41b0 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): dump: 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.295 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc84001770 con 0x7efc900a49a0 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): dump: 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.296 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3849895979 0 0) 0x7efc900a0f60 con 0x7efc90137940 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efc900a1f30 con 0x7efc90137940 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc88001890 con 0x7efc90137940 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:53:53.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient(hunting): dump: 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.298 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3074518644 0 0) 0x7efc901396b0 con 0x7efc900a41b0 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.7:6791/0 conn(0x7efc900a49a0 legacy=0x7efc901d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.7:6789/0 conn(0x7efc90137940 legacy=0x7efc901d4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 monclient: found mon.b 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc90138e30 con 0x7efc900a41b0 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc9588c640 5 monclient: authenticate success, global_id 95933 2024-05-08T15:53:53.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc9588c640 10 monclient: _renew_subs 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc9588c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7efc901393d0 con 0x7efc900a41b0 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc9588c640 10 monclient: _renew_subs 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc9588c640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7efc90141630 con 0x7efc900a41b0 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efc80004590 con 0x7efc900a41b0 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc80004b00 con 0x7efc900a41b0 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc8cff9640 4 set_mon_vals no callback set 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.282+0000 7efc77fff640 10 monclient: dump: 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.300 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.283+0000 7efc9588c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.283+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7efc80004ca0 con 0x7efc900a41b0 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.283+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efc700065c0 con 0x7efc70002860 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.283+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(301..301 src has 1..301) ==== 3506+0+0 (unknown 1436118537 0 0) 0x7efc80046cc0 con 0x7efc900a41b0 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.283+0000 7efc8f7fe640 10 monclient: get_auth_request con 0x7efc7c03a400 auth_method 0 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2889101031 0 0) 0x7efc700065c0 con 0x7efc70002860 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient(hunting): my global_id is 76665 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc901396b0 con 0x7efc70002860 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3737175977 0 0) 0x7efc901396b0 con 0x7efc70002860 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efc700065c0 con 0x7efc70002860 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc84004ff0 con 0x7efc70002860 2024-05-08T15:53:53.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient: dump: 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.303 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1483852719 0 0) 0x7efc700065c0 con 0x7efc70002860 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc7000a150 con 0x7efc70002860 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7efc840072d0 con 0x7efc70002860 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.81:6790/0 conn(0x7efc70002860 legacy=0x7efc70004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.284+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.81:6790/0 conn(0x7efc70002860 legacy=0x7efc70004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.291+0000 7efc9588c640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.291+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efc70009e40 con 0x7efc70009100 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.291+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1311666615 0 0) 0x7efc70009e40 con 0x7efc70009100 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient(hunting): my global_id is 76674 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc700065c0 con 0x7efc70009100 2024-05-08T15:53:53.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4226744643 0 0) 0x7efc700065c0 con 0x7efc70009100 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efc70009e40 con 0x7efc70009100 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efc84006b90 con 0x7efc70009100 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient: dump: 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.307 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 444638543 0 0) 0x7efc70009e40 con 0x7efc70009100 2024-05-08T15:53:53.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efc7000c3d0 con 0x7efc70009100 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7682 (unknown 0 0 1143418405) 0x7efc84006d70 con 0x7efc70009100 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.81:6790/0 conn(0x7efc70009100 legacy=0x7efc70009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.292+0000 7efc77fff640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.81:6790/0 conn(0x7efc70009100 legacy=0x7efc70009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:53.309 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 425582, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:53.310 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.311 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.312 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:53.313 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.314 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.315 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:53.316 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.317 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.318 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.319 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.320 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.321 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.322 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.323 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.324 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.325 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:53.326 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.327 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.328 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.328 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:53:53.328 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.328 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.328 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.328 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:53.328 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:53.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.81:6812/847627118 conn(0x7efc7c03a400 legacy=0x7efc7c03c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 10 monclient: shutdown 2024-05-08T15:53:53.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 >> v1:172.21.15.7:6790/0 conn(0x7efc900a41b0 legacy=0x7efc90140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc8f7fe640 1 -- 172.21.15.7:0/2476417351 reap_dead start 2024-05-08T15:53:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc8f7fe640 1 -- 172.21.15.7:0/2476417351 reap_dead start 2024-05-08T15:53:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 shutdown_connections 2024-05-08T15:53:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 >> 172.21.15.7:0/2476417351 conn(0x7efc900a9fa0 msgr2=0x7efc900a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 shutdown_connections 2024-05-08T15:53:53.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.293+0000 7efc9588c640 1 -- 172.21.15.7:0/2476417351 wait complete. 2024-05-08T15:53:53.330 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:53.330 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon h 2024-05-08T15:53:53.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:53:53.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.438+0000 7f53e0dab640 10 monclient: get_monmap_and_config 2024-05-08T15:53:53.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.438+0000 7f53e0dab640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.438+0000 7f53e0dab640 10 monclient: monmap: 2024-05-08T15:53:53.434 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:53.439780+0000 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:53.439780+0000 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.435 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.439+0000 7f53e0dab640 1 Processor -- start 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.439+0000 7f53e0dab640 1 -- start start 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.439+0000 7f53e0dab640 10 monclient: init 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient: _add_conns ranks=[4,2,3,7,8,1,0,6,5] 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient(hunting): picked mon.noname-c con 0x7f53dc137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient(hunting): picked mon.noname-b con 0x7f53dc0a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient(hunting): picked mon.noname-g con 0x7f53dc0a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:53.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc0a2c70 con 0x7f53dc0a49a0 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc0a2f50 con 0x7f53dc0a41b0 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc0a1670 con 0x7f53dc137940 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient(hunting): authenticate will time out at 1241.554806s 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d9d9b640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f53dc0a49a0 0x7f53dc13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43696/0 (socket says 172.21.15.7:43696) 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d9d9b640 1 -- 172.21.15.7:0/2962295925 learned_addr learned my addr 172.21.15.7:0/2962295925 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53da59c640 1 --1- 172.21.15.7:0/2962295925 >> v1:172.21.15.7:6791/0 conn(0x7f53dc137940 0x7f53dc137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:52386/0 (socket says 172.21.15.7:52386) 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1809517011 0 0) 0x7f53dc0a2c70 con 0x7f53dc0a49a0 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): my global_id is 95942 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c8003520 con 0x7f53dc0a49a0 2024-05-08T15:53:53.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 515048995 0 0) 0x7f53dc0a1670 con 0x7f53dc137940 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient(hunting): my global_id is 124339 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53dc0a2c70 con 0x7f53dc137940 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3444997606 0 0) 0x7f53c8003520 con 0x7f53dc0a49a0 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 >> v1:172.21.15.7:6791/0 conn(0x7f53dc137940 legacy=0x7f53dc137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 >> v1:172.21.15.81:6790/0 conn(0x7f53dc0a41b0 legacy=0x7f53dc140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 monclient: found mon.noname-b 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc0a39f0 con 0x7f53dc0a49a0 2024-05-08T15:53:53.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53c40017b0 con 0x7f53dc0a49a0 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 5 monclient: authenticate success, global_id 95942 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: dump: 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.448 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f53c4001a90 con 0x7f53dc0a49a0 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53e0dab640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.440+0000 7f53d959a640 1 -- 172.21.15.7:0/2962295925 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53c4004be0 con 0x7f53dc0a49a0 2024-05-08T15:53:53.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 4 set_mon_vals no callback set 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53d959a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53d959a640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53d959a640 10 monclient: dump: 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.450 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 10 monclient: shutdown 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 1 -- 172.21.15.7:0/2962295925 >> v1:172.21.15.7:6790/0 conn(0x7f53dc0a49a0 legacy=0x7f53dc13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 1 -- 172.21.15.7:0/2962295925 shutdown_connections 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 1 -- 172.21.15.7:0/2962295925 >> 172.21.15.7:0/2962295925 conn(0x7f53dc0a9fa0 msgr2=0x7f53dc0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 1 -- 172.21.15.7:0/2962295925 shutdown_connections 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.441+0000 7f53e0dab640 1 -- 172.21.15.7:0/2962295925 wait complete. 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.442+0000 7f53e0dab640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.442+0000 7f53e0dab640 10 monclient: monmap: 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.452 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.442+0000 7f53e0dab640 1 Processor -- start 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.442+0000 7f53e0dab640 1 -- start start 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.442+0000 7f53e0dab640 10 monclient: init 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 10 monclient: _add_conns ranks=[6,7,1,5,4,0,8,2,3] 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 10 monclient(hunting): picked mon.noname-d con 0x7f53dc0a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 10 monclient(hunting): picked mon.noname-i con 0x7f53dc0a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 10 monclient(hunting): picked mon.noname-f con 0x7f53dc137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc0a0f60 con 0x7f53dc137940 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc1356e0 con 0x7f53dc0a41b0 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53dc0a1f30 con 0x7f53dc0a49a0 2024-05-08T15:53:53.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53e0dab640 10 monclient(hunting): authenticate will time out at 1241.557823s 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53da59c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f53dc0a41b0 0x7f53dc1360c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:43036/0 (socket says 172.21.15.7:43036) 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53da59c640 1 -- 172.21.15.7:0/2771554775 learned_addr learned my addr 172.21.15.7:0/2771554775 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2126175712 0 0) 0x7f53dc1356e0 con 0x7f53dc0a41b0 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): my global_id is 124422 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53bc003520 con 0x7f53dc0a41b0 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 244844256 0 0) 0x7f53dc0a1f30 con 0x7f53dc0a49a0 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): my global_id is 115063 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53dc1356e0 con 0x7f53dc0a49a0 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1368898159 0 0) 0x7f53d4001230 con 0x7f53dc137940 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): my global_id is 6328 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53dc0a1f30 con 0x7f53dc137940 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2756165330 0 0) 0x7f53bc003520 con 0x7f53dc0a41b0 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53d4001230 con 0x7f53dc0a41b0 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53cc001d00 con 0x7f53dc0a41b0 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:53.455 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.456 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1970051157 0 0) 0x7f53dc1356e0 con 0x7f53dc0a49a0 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53bc003520 con 0x7f53dc0a49a0 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53c4004b20 con 0x7f53dc0a49a0 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.457 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1425076603 0 0) 0x7f53dc0a1f30 con 0x7f53dc137940 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53dc1356e0 con 0x7f53dc137940 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53d4001ba0 con 0x7f53dc137940 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient(hunting): dump: 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.459 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1403228132 0 0) 0x7f53d4001230 con 0x7f53dc0a41b0 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.81:6792/0 conn(0x7f53dc0a49a0 legacy=0x7f53dc1d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.81:6789/0 conn(0x7f53dc137940 legacy=0x7f53dc1d4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 1 monclient: found mon.d 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.443+0000 7f53d2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53dc134020 con 0x7f53dc0a41b0 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53e0dab640 5 monclient: authenticate success, global_id 124422 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53e0dab640 10 monclient: _renew_subs 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53e0dab640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f53dc1d5a90 con 0x7f53dc0a41b0 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53e0dab640 10 monclient: _renew_subs 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53e0dab640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f53dc1d5e40 con 0x7f53dc0a41b0 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f53cc002040 con 0x7f53dc0a41b0 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53cc005150 con 0x7f53dc0a41b0 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d8d99640 4 set_mon_vals no callback set 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 10 monclient: dump: 2024-05-08T15:53:53.461 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.462 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.444+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f53cc0104d0 con 0x7f53dc0a41b0 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53e0dab640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f53c00065c0 con 0x7f53c0002860 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(301..301 src has 1..301) ==== 3506+0+0 (unknown 1436118537 0 0) 0x7f53cc046eb0 con 0x7f53dc0a41b0 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d9d9b640 10 monclient: get_auth_request con 0x7f53bc03a410 auth_method 0 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4274861535 0 0) 0x7f53c00065c0 con 0x7f53c0002860 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 10 monclient(hunting): my global_id is 6548 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53d4001230 con 0x7f53c0002860 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 4122965467 0 0) 0x7f53d4001230 con 0x7f53c0002860 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53c00065c0 con 0x7f53c0002860 2024-05-08T15:53:53.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53d4003b20 con 0x7f53c0002860 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 10 monclient: dump: 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.464 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1638979129 0 0) 0x7f53c00065c0 con 0x7f53c0002860 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.445+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53c000a150 con 0x7f53c0002860 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.446+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f53d4003ed0 con 0x7f53c0002860 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.446+0000 7f53d2ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.446+0000 7f53d2ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.446+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.81:6791/0 conn(0x7f53c0002860 legacy=0x7f53c0004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.446+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.81:6791/0 conn(0x7f53c0002860 legacy=0x7f53c0004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.452+0000 7f53e0dab640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.452+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f53c0009e40 con 0x7f53c0009100 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991553784 0 0) 0x7f53c0009e40 con 0x7f53c0009100 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 10 monclient(hunting): my global_id is 6557 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53c00065c0 con 0x7f53c0009100 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2511295939 0 0) 0x7f53c00065c0 con 0x7f53c0009100 2024-05-08T15:53:53.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f53c0009e40 con 0x7f53c0009100 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f53d40046b0 con 0x7f53c0009100 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 10 monclient: dump: 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.468 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2793619003 0 0) 0x7f53c0009e40 con 0x7f53c0009100 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.453+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f53c000c3d0 con 0x7f53c0009100 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7682 (unknown 0 0 1319374504) 0x7f53d40059d0 con 0x7f53c0009100 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53d2ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53d2ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.81:6791/0 conn(0x7f53c0009100 legacy=0x7f53c0009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53d2ffd640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.81:6791/0 conn(0x7f53c0009100 legacy=0x7f53c0009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 425759, 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:53.470 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.471 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.476 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.477 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.478 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:53.479 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.480 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:53.481 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.482 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.483 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.484 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:53.485 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.486 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.487 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.488 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:53.488 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.81:6812/847627118 conn(0x7f53bc03a410 legacy=0x7f53bc03c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 10 monclient: shutdown 2024-05-08T15:53:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 >> v1:172.21.15.7:6792/0 conn(0x7f53dc0a41b0 legacy=0x7f53dc1360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53dad9d640 1 -- 172.21.15.7:0/2771554775 reap_dead start 2024-05-08T15:53:53.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53dad9d640 1 -- 172.21.15.7:0/2771554775 reap_dead start 2024-05-08T15:53:53.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 shutdown_connections 2024-05-08T15:53:53.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 >> 172.21.15.7:0/2771554775 conn(0x7f53dc0a9fa0 msgr2=0x7f53dc0a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 shutdown_connections 2024-05-08T15:53:53.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.454+0000 7f53e0dab640 1 -- 172.21.15.7:0/2771554775 wait complete. 2024-05-08T15:53:53.490 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:53.490 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 8 for mon i 2024-05-08T15:53:53.491 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:53:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.599+0000 7f0eaef75640 10 monclient: get_monmap_and_config 2024-05-08T15:53:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.599+0000 7f0eaef75640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.599+0000 7f0eaef75640 10 monclient: monmap: 2024-05-08T15:53:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.595 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:53:53.600698+0000 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:53:53.600698+0000 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.596 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 1 Processor -- start 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 1 -- start start 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient: init 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient: _add_conns ranks=[8,5,2,4,6,7,1,0,3] 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient(hunting): picked mon.noname-e con 0x7f0ea8137940 addr v1:172.21.15.7:6793/0 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient(hunting): picked mon.noname-h con 0x7f0ea80a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient(hunting): picked mon.noname-b con 0x7f0ea80a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80a2c70 con 0x7f0ea80a41b0 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80a2f50 con 0x7f0ea80a49a0 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80a1670 con 0x7f0ea8137940 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eaef75640 10 monclient(hunting): authenticate will time out at 1241.715604s 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.600+0000 7f0eacd11640 1 -- >> v1:172.21.15.7:6793/0 conn(0x7f0ea8137940 legacy=0x7f0ea8137d20 unknown :-1 s=STATE_CONNECTING_RE l=0).process reconnect failed to v1:172.21.15.7:6793/0 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ead512640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f0ea80a41b0 0x7f0ea8140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43704/0 (socket says 172.21.15.7:43704) 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ead512640 1 -- 172.21.15.7:0/3176016980 learned_addr learned my addr 172.21.15.7:0/3176016980 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429392652 0 0) 0x7f0ea80a2c70 con 0x7f0ea80a41b0 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): my global_id is 95951 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e98003520 con 0x7f0ea80a41b0 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 487747293 0 0) 0x7f0ea80a2f50 con 0x7f0ea80a49a0 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient(hunting): my global_id is 6566 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea80a2c70 con 0x7f0ea80a49a0 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2416951613 0 0) 0x7f0e98003520 con 0x7f0ea80a41b0 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 >> v1:172.21.15.7:6793/0 conn(0x7f0ea8137940 legacy=0x7f0ea8137d20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 >> v1:172.21.15.81:6791/0 conn(0x7f0ea80a49a0 legacy=0x7f0ea813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 monclient: found mon.noname-b 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea80a39f0 con 0x7f0ea80a41b0 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ea0001b30 con 0x7f0ea80a41b0 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0eaef75640 5 monclient: authenticate success, global_id 95951 2024-05-08T15:53:53.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0eaef75640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: dump: 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.609 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0eaef75640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0ea0001e50 con 0x7f0ea80a41b0 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 1 -- 172.21.15.7:0/3176016980 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ea0004f60 con 0x7f0ea80a41b0 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0ea77fe640 10 monclient: dump: 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.610 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0eaef75640 10 monclient: get_monmap_and_config success 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.601+0000 7f0eaef75640 4 set_mon_vals no callback set 2024-05-08T15:53:53.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 10 monclient: shutdown 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 1 -- 172.21.15.7:0/3176016980 >> v1:172.21.15.7:6790/0 conn(0x7f0ea80a41b0 legacy=0x7f0ea8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 1 -- 172.21.15.7:0/3176016980 shutdown_connections 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 1 -- 172.21.15.7:0/3176016980 >> 172.21.15.7:0/3176016980 conn(0x7f0ea80a9fa0 msgr2=0x7f0ea80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 1 -- 172.21.15.7:0/3176016980 shutdown_connections 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 1 -- 172.21.15.7:0/3176016980 wait complete. 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 10 monclient: build_initial_monmap 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.602+0000 7f0eaef75640 10 monclient: monmap: 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:53:53.612 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.603+0000 7f0eaef75640 1 Processor -- start 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.603+0000 7f0eaef75640 1 -- start start 2024-05-08T15:53:53.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.603+0000 7f0eaef75640 10 monclient: init 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.603+0000 7f0eaef75640 10 monclient: _reopen_session rank -1 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.603+0000 7f0eaef75640 10 monclient: _add_conns ranks=[3,7,2,1,6,5,4,0,8] 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.603+0000 7f0eaef75640 10 monclient(hunting): picked mon.noname-g con 0x7f0ea80a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.603+0000 7f0eaef75640 10 monclient(hunting): picked mon.noname-i con 0x7f0ea80a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 10 monclient(hunting): picked mon.noname-b con 0x7f0ea8137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80a0f60 con 0x7f0ea8137940 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea8133610 con 0x7f0ea80a41b0 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea81350d0 con 0x7f0ea80a49a0 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 10 monclient(hunting): _renew_subs 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 10 monclient(hunting): authenticate will time out at 1241.718713s 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ead512640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f0ea8137940 0x7f0ea81d4d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:43718/0 (socket says 172.21.15.7:43718) 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ead512640 1 -- 172.21.15.7:0/2583380821 learned_addr learned my addr 172.21.15.7:0/2583380821 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388755273 0 0) 0x7f0ea80a0f60 con 0x7f0ea8137940 2024-05-08T15:53:53.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): my global_id is 95960 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e8c003520 con 0x7f0ea8137940 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3256262020 0 0) 0x7f0ea8133610 con 0x7f0ea80a41b0 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): my global_id is 76692 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea80a0f60 con 0x7f0ea80a41b0 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1822925081 0 0) 0x7f0ea81350d0 con 0x7f0ea80a49a0 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): my global_id is 115072 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea8133610 con 0x7f0ea80a49a0 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2067577706 0 0) 0x7f0e8c003520 con 0x7f0ea8137940 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ea81350d0 con 0x7f0ea8137940 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ea0004ea0 con 0x7f0ea8137940 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): dump: 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.616 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 528686888 0 0) 0x7f0ea8133610 con 0x7f0ea80a49a0 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e8c003520 con 0x7f0ea80a49a0 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0e94001a50 con 0x7f0ea80a49a0 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:53:53.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient(hunting): dump: 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.618 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 582965100 0 0) 0x7f0ea80a0f60 con 0x7f0ea80a41b0 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0ea8133610 con 0x7f0ea80a41b0 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 120149826 0 0) 0x7f0ea81350d0 con 0x7f0ea8137940 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.81:6792/0 conn(0x7f0ea80a49a0 legacy=0x7f0ea81d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.81:6790/0 conn(0x7f0ea80a41b0 legacy=0x7f0ea8136ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient: _finish_hunting 0 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 monclient: found mon.b 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea81353b0 con 0x7f0ea8137940 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient: _finish_auth 0 2024-05-08T15:53:53.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 10 monclient: _check_auth_tickets 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0ea57fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 5 monclient: authenticate success, global_id 95960 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 10 monclient: _renew_subs 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea81ccd20 con 0x7f0ea8137940 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 10 monclient: _renew_subs 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.604+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f0ea8134300 con 0x7f0ea8137940 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0ea0001bd0 con 0x7f0ea8137940 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea57fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0ea0005520 con 0x7f0ea8137940 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea67fc640 4 set_mon_vals no callback set 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea57fa640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:53:53.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea57fa640 10 monclient: dump: 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.621 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0eaef75640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0e880065c0 con 0x7f0e88002860 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.605+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f0ea00108a0 con 0x7f0ea8137940 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(301..301 src has 1..301) ==== 3506+0+0 (unknown 1436118537 0 0) 0x7f0ea00471a0 con 0x7f0ea8137940 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4021827088 0 0) 0x7f0e880065c0 con 0x7f0e88002860 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 10 monclient(hunting): my global_id is 115081 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ea81350d0 con 0x7f0e88002860 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea7fff640 10 monclient: get_auth_request con 0x7f0e8c03a270 auth_method 0 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2616381438 0 0) 0x7f0ea81350d0 con 0x7f0e88002860 2024-05-08T15:53:53.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e880065c0 con 0x7f0e88002860 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0e9c0049c0 con 0x7f0e88002860 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 10 monclient: dump: 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.623 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 784707979 0 0) 0x7f0e880065c0 con 0x7f0e88002860 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.606+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e8800a150 con 0x7f0e88002860 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.607+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f0e9c004df0 con 0x7f0e88002860 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.607+0000 7f0ea57fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.607+0000 7f0ea57fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.607+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.81:6792/0 conn(0x7f0e88002860 legacy=0x7f0e88004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.607+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.81:6792/0 conn(0x7f0e88002860 legacy=0x7f0e88004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.614+0000 7f0eaef75640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.614+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f0e88009e00 con 0x7f0e880090e0 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322510213 0 0) 0x7f0e88009e00 con 0x7f0e880090e0 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 10 monclient(hunting): _negotiate 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 10 monclient(hunting): my global_id is 115090 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0e880065c0 con 0x7f0e880090e0 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3533024517 0 0) 0x7f0e880065c0 con 0x7f0e880090e0 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0e88009e00 con 0x7f0e880090e0 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0e9c005740 con 0x7f0e880090e0 2024-05-08T15:53:53.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 10 monclient: dump: 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:53:53.627 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3602665817 0 0) 0x7f0e88009e00 con 0x7f0e880090e0 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.615+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0e8800c380 con 0x7f0e880090e0 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7682 (unknown 0 0 2622256301) 0x7f0e9c005a20 con 0x7f0e880090e0 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0ea57fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0ea57fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.81:6792/0 conn(0x7f0e880090e0 legacy=0x7f0e88009550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0ea57fa640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.81:6792/0 conn(0x7f0e880090e0 legacy=0x7f0e88009550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:53:53.628 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 56, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 168479, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:53:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: 7 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.630 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.631 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:53:53.632 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:53:53.633 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.634 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.635 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:53:53.636 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.637 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.638 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.639 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.640 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.641 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.642 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.643 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:53:53.644 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.645 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:53:53.646 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:53:53.647 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:53:53.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.81:6812/847627118 conn(0x7f0e8c03a270 legacy=0x7f0e8c03c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 10 monclient: shutdown 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 >> v1:172.21.15.7:6790/0 conn(0x7f0ea8137940 legacy=0x7f0ea81d4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0ead512640 1 -- 172.21.15.7:0/2583380821 reap_dead start 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0ead512640 1 -- 172.21.15.7:0/2583380821 reap_dead start 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 shutdown_connections 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 >> 172.21.15.7:0/2583380821 conn(0x7f0ea80a9fa0 msgr2=0x7f0ea81308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 shutdown_connections 2024-05-08T15:53:53.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:53:53.616+0000 7f0eaef75640 1 -- 172.21.15.7:0/2583380821 wait complete. 2024-05-08T15:53:53.649 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 8 2024-05-08T15:53:53.649 INFO:tasks.mon_thrash.mon_thrasher:waiting for 90.0 secs before reviving monitors 2024-05-08T15:53:54.870 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:54.870 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39016]: starting. 2024-05-08T15:53:54.870 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39016]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-38977 2024-05-08T15:53:54.870 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39016]: created object 0... 2024-05-08T15:53:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39016]: created object 25... 2024-05-08T15:53:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39016]: created object 49... 2024-05-08T15:53:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39016]: finishing. 2024-05-08T15:53:54.871 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39016]: shutting down. 2024-05-08T15:53:54.878 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:54.878 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39018]: starting. 2024-05-08T15:53:54.878 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39018]: listing objects. 2024-05-08T15:53:54.878 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39018]: listed object 0... 2024-05-08T15:53:54.878 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39018]: listed object 25... 2024-05-08T15:53:54.878 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39018]: saw 50 objects 2024-05-08T15:53:54.878 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39018]: shutting down. 2024-05-08T15:53:55.794 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:55.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39017]: starting. 2024-05-08T15:53:55.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39017]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-38977 2024-05-08T15:53:55.794 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39017]: shutting down. 2024-05-08T15:53:55.796 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:55.797 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:53:55.797 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:53:55.799 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:53:57.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39087]: starting. 2024-05-08T15:53:57.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39087]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-39086 2024-05-08T15:53:57.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39087]: created object 0... 2024-05-08T15:53:57.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39087]: created object 25... 2024-05-08T15:53:57.891 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39087]: created object 49... 2024-05-08T15:53:57.892 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39087]: finishing. 2024-05-08T15:53:57.892 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39087]: shutting down. 2024-05-08T15:53:58.815 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39088]: starting. 2024-05-08T15:53:58.816 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39088]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-39086 2024-05-08T15:53:58.816 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39088]: shutting down. 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39135]: starting. 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39135]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-39086 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39135]: created object 0... 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39135]: created object 25... 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39135]: created object 49... 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39135]: finishing. 2024-05-08T15:54:00.927 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39135]: shutting down. 2024-05-08T15:54:00.939 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:00.939 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39137]: starting. 2024-05-08T15:54:00.939 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39137]: listing objects. 2024-05-08T15:54:00.939 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39137]: listed object 0... 2024-05-08T15:54:00.939 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39137]: listed object 25... 2024-05-08T15:54:00.939 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39137]: saw 50 objects 2024-05-08T15:54:00.939 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39137]: shutting down. 2024-05-08T15:54:01.845 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:01.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39136]: starting. 2024-05-08T15:54:01.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39136]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-39086 2024-05-08T15:54:01.846 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39136]: shutting down. 2024-05-08T15:54:01.847 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:01.847 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:01.847 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:54:01.851 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:54:03.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39206]: starting. 2024-05-08T15:54:03.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39206]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-39205 2024-05-08T15:54:03.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39206]: created object 0... 2024-05-08T15:54:03.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39206]: created object 25... 2024-05-08T15:54:03.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39206]: created object 49... 2024-05-08T15:54:03.862 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39206]: finishing. 2024-05-08T15:54:03.863 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39206]: shutting down. 2024-05-08T15:54:04.785 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39207]: starting. 2024-05-08T15:54:04.786 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39207]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-39205 2024-05-08T15:54:04.786 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39207]: shutting down. 2024-05-08T15:54:06.873 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:06.873 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39242]: starting. 2024-05-08T15:54:06.873 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39242]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-39205 2024-05-08T15:54:06.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39242]: created object 0... 2024-05-08T15:54:06.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39242]: created object 25... 2024-05-08T15:54:06.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39242]: created object 49... 2024-05-08T15:54:06.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39242]: finishing. 2024-05-08T15:54:06.874 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39242]: shutting down. 2024-05-08T15:54:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39244]: starting. 2024-05-08T15:54:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39244]: listing objects. 2024-05-08T15:54:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39244]: listed object 0... 2024-05-08T15:54:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39244]: listed object 25... 2024-05-08T15:54:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39244]: saw 50 objects 2024-05-08T15:54:06.884 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39244]: shutting down. 2024-05-08T15:54:07.790 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:07.790 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39243]: starting. 2024-05-08T15:54:07.790 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39243]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-39205 2024-05-08T15:54:07.790 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39243]: shutting down. 2024-05-08T15:54:07.792 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:07.792 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:07.792 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:54:07.797 DEBUG:teuthology.orchestra.run.smithi081:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel 2024-05-08T15:54:09.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39313]: starting. 2024-05-08T15:54:09.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39313]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-39312 2024-05-08T15:54:09.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39313]: created object 0... 2024-05-08T15:54:09.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39313]: created object 25... 2024-05-08T15:54:09.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39313]: created object 49... 2024-05-08T15:54:09.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39313]: finishing. 2024-05-08T15:54:09.911 INFO:teuthology.orchestra.run.smithi081.stdout:process_1_[39313]: shutting down. 2024-05-08T15:54:10.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39314]: starting. 2024-05-08T15:54:10.838 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39314]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-39312 2024-05-08T15:54:10.839 INFO:teuthology.orchestra.run.smithi081.stdout:process_2_[39314]: shutting down. 2024-05-08T15:54:12.878 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39349]: starting. 2024-05-08T15:54:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39349]: creating pool ceph_test_rados_delete_pools_parallel.smithi081-39312 2024-05-08T15:54:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39349]: created object 0... 2024-05-08T15:54:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39349]: created object 25... 2024-05-08T15:54:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39349]: created object 49... 2024-05-08T15:54:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39349]: finishing. 2024-05-08T15:54:12.879 INFO:teuthology.orchestra.run.smithi081.stdout:process_3_[39349]: shutting down. 2024-05-08T15:54:12.890 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:12.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39351]: starting. 2024-05-08T15:54:12.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39351]: listing objects. 2024-05-08T15:54:12.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39351]: listed object 0... 2024-05-08T15:54:12.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39351]: listed object 25... 2024-05-08T15:54:12.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39351]: saw 50 objects 2024-05-08T15:54:12.890 INFO:teuthology.orchestra.run.smithi081.stdout:process_5_[39351]: shutting down. 2024-05-08T15:54:13.768 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:13.768 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39350]: starting. 2024-05-08T15:54:13.768 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39350]: deleting pool ceph_test_rados_delete_pools_parallel.smithi081-39312 2024-05-08T15:54:13.769 INFO:teuthology.orchestra.run.smithi081.stdout:process_4_[39350]: shutting down. 2024-05-08T15:54:13.770 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:13.770 INFO:teuthology.orchestra.run.smithi081.stdout:******************************* 2024-05-08T15:54:13.770 INFO:teuthology.orchestra.run.smithi081.stdout:******* SUCCESS ********** 2024-05-08T15:54:13.772 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-05-08T15:54:13.781 INFO:tasks.mon_thrash:joining mon_thrasher 2024-05-08T15:55:23.651 INFO:tasks.mon_thrash.mon_thrasher:killing mon.e 2024-05-08T15:55:23.651 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.e 2024-05-08T15:55:23.651 INFO:tasks.ceph.mon.e:Restarting daemon 2024-05-08T15:55:23.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i e 2024-05-08T15:55:23.655 INFO:tasks.ceph.mon.e:Started 2024-05-08T15:55:23.655 INFO:tasks.mon_thrash:Sending STOP to mon a 2024-05-08T15:55:23.655 INFO:tasks.ceph.mon.a:Sent signal 19 2024-05-08T15:55:23.655 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-05-08T15:55:38.656 INFO:tasks.mon_thrash:Sending CONT to mon a 2024-05-08T15:55:38.656 INFO:tasks.ceph.mon.a:Sent signal 18 2024-05-08T15:55:38.656 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:55:38.657 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:55:38.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.786+0000 7f12ed11c640 10 monclient: get_monmap_and_config 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.787+0000 7f12ed11c640 10 monclient: build_initial_monmap 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.787+0000 7f12ed11c640 10 monclient: monmap: 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:38.788004+0000 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:38.788004+0000 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:38.783 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:38.784 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:38.784 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.787+0000 7f12ed11c640 1 Processor -- start 2024-05-08T15:55:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.787+0000 7f12ed11c640 1 -- start start 2024-05-08T15:55:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.787+0000 7f12ed11c640 10 monclient: init 2024-05-08T15:55:38.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 10 monclient: _add_conns ranks=[0,6,3,2,4,8,1,7,5] 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 10 monclient(hunting): picked mon.noname-a con 0x7f12e8137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 10 monclient(hunting): picked mon.noname-d con 0x7f12e80a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 10 monclient(hunting): picked mon.noname-g con 0x7f12e80a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80a2c70 con 0x7f12e8137940 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80a2f50 con 0x7f12e80a41b0 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80a1670 con 0x7f12e80a49a0 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 10 monclient(hunting): authenticate will time out at 1346.902875s 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e659c640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f12e80a49a0 0x7f12e813cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35146/0 (socket says 172.21.15.7:35146) 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e659c640 1 -- 172.21.15.7:0/1000101025 learned_addr learned my addr 172.21.15.7:0/1000101025 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e6d9d640 1 --1- 172.21.15.7:0/1000101025 >> v1:172.21.15.7:6789/0 conn(0x7f12e8137940 0x7f12e8137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:33984/0 (socket says 172.21.15.7:33984) 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 377999747 0 0) 0x7f12e80a2c70 con 0x7f12e8137940 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:55:38.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): my global_id is 95814 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12d4003520 con 0x7f12e8137940 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209368422 0 0) 0x7f12e80a1670 con 0x7f12e80a49a0 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): my global_id is 124548 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12e80a2c70 con 0x7f12e80a49a0 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3318647944 0 0) 0x7f12e80a2f50 con 0x7f12e80a41b0 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _negotiate 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:38.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient(hunting): my global_id is 76872 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12e80a1670 con 0x7f12e80a41b0 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462100112 0 0) 0x7f12d4003520 con 0x7f12e8137940 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 >> v1:172.21.15.7:6792/0 conn(0x7f12e80a49a0 legacy=0x7f12e813cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 >> v1:172.21.15.81:6790/0 conn(0x7f12e80a41b0 legacy=0x7f12e8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient: _finish_hunting 0 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 monclient: found mon.noname-a 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80a39f0 con 0x7f12e8137940 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient: _finish_auth 0 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient: _check_auth_tickets 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f12e0001b30 con 0x7f12e8137940 2024-05-08T15:55:38.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 5 monclient: authenticate success, global_id 95814 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12ed11c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.788+0000 7f12e5d9b640 10 monclient: dump: 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:38.796 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f12e0001e50 con 0x7f12e8137940 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12e5d9b640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12e5d9b640 1 -- 172.21.15.7:0/1000101025 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f12e00020c0 con 0x7f12e8137940 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12e5d9b640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12e5d9b640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12e5d9b640 10 monclient: dump: 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:38.797 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:38.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 4 set_mon_vals no callback set 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 10 monclient: shutdown 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 1 -- 172.21.15.7:0/1000101025 >> v1:172.21.15.7:6789/0 conn(0x7f12e8137940 legacy=0x7f12e8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 1 -- 172.21.15.7:0/1000101025 shutdown_connections 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 1 -- 172.21.15.7:0/1000101025 >> 172.21.15.7:0/1000101025 conn(0x7f12e80a9fa0 msgr2=0x7f12e80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 1 -- 172.21.15.7:0/1000101025 shutdown_connections 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.789+0000 7f12ed11c640 1 -- 172.21.15.7:0/1000101025 wait complete. 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.790+0000 7f12ed11c640 10 monclient: build_initial_monmap 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.790+0000 7f12ed11c640 10 monclient: monmap: 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:38.799 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.790+0000 7f12ed11c640 1 Processor -- start 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.790+0000 7f12ed11c640 1 -- start start 2024-05-08T15:55:38.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.790+0000 7f12ed11c640 10 monclient: init 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 10 monclient: _add_conns ranks=[6,8,1,2,5,0,4,3,7] 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 10 monclient(hunting): picked mon.noname-d con 0x7f12e80a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 10 monclient(hunting): picked mon.noname-e con 0x7f12e80a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 10 monclient(hunting): picked mon.noname-f con 0x7f12e8137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80a1f30 con 0x7f12e8137940 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e80716c0 con 0x7f12e80a41b0 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12e806f340 con 0x7f12e80a49a0 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12ed11c640 10 monclient(hunting): authenticate will time out at 1346.906147s 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12e6d9d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f12e80a41b0 0x7f12e806e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35148/0 (socket says 172.21.15.7:35148) 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12e6d9d640 1 -- 172.21.15.7:0/3054280281 learned_addr learned my addr 172.21.15.7:0/3054280281 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3771406002 0 0) 0x7f12e80716c0 con 0x7f12e80a41b0 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:38.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): my global_id is 124557 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12c8003520 con 0x7f12e80a41b0 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219093048 0 0) 0x7f12e806f340 con 0x7f12e80a49a0 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 10 monclient(hunting): my global_id is 144107 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.791+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12e80716c0 con 0x7f12e80a49a0 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3215647429 0 0) 0x7f12c8003520 con 0x7f12e80a41b0 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f12e806f340 con 0x7f12e80a41b0 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f12e0002000 con 0x7f12e80a41b0 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:38.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): dump: 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:38.803 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477912652 0 0) 0x7f12e80a1670 con 0x7f12e8137940 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): my global_id is 6400 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f12c8003520 con 0x7f12e8137940 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1231448995 0 0) 0x7f12e80716c0 con 0x7f12e80a49a0 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f12e80a1670 con 0x7f12e80a49a0 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f12d0001450 con 0x7f12e80a49a0 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:38.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient(hunting): dump: 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:38.805 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2416986407 0 0) 0x7f12e806f340 con 0x7f12e80a41b0 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 >> v1:172.21.15.7:6793/0 conn(0x7f12e80a49a0 legacy=0x7f12e806ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 >> v1:172.21.15.81:6789/0 conn(0x7f12e8137940 legacy=0x7f12e81d8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 monclient: found mon.d 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80703a0 con 0x7f12e80a41b0 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: _finish_auth 0 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12ed11c640 5 monclient: authenticate success, global_id 124557 2024-05-08T15:55:38.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12ed11c640 10 monclient: _renew_subs 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12ed11c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f12e806f6d0 con 0x7f12e80a41b0 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12ed11c640 10 monclient: _renew_subs 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12ed11c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f12e806fd90 con 0x7f12e80a41b0 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f12e0005150 con 0x7f12e80a41b0 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f12e0005640 con 0x7f12e80a41b0 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12e559a640 4 set_mon_vals no callback set 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.792+0000 7f12df7fe640 10 monclient: dump: 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:38.807 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.793+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f12e00109c0 con 0x7f12e80a41b0 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.793+0000 7f12ed11c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:38.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.793+0000 7f12ed11c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.793+0000 7f12ed11c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.793+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc005080 con 0x7f12e80a41b0 2024-05-08T15:55:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.793+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f12e00471b0 con 0x7f12e80a41b0 2024-05-08T15:55:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.793+0000 7f12e659c640 10 monclient: get_auth_request con 0x7f12c803a240 auth_method 0 2024-05-08T15:55:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.798+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f12e0016400 con 0x7f12e80a41b0 2024-05-08T15:55:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.798+0000 7f12df7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:38.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:38.798+0000 7f12df7fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.057+0000 7f12ed11c640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:55:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.057+0000 7f12ed11c640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:55:39.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.057+0000 7f12ed11c640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:39.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.057+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f12cc005370 con 0x7f12e80a41b0 2024-05-08T15:55:39.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.057+0000 7f12df7fe640 1 -- 172.21.15.7:0/3054280281 <== mon.6 v1:172.21.15.7:6792/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 365228737) 0x7f12e0015d00 con 0x7f12e80a41b0 2024-05-08T15:55:39.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.057+0000 7f12df7fe640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:55:39.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.057+0000 7f12df7fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:39.053 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":60,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:55:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 >> v1:172.21.15.81:6812/847627118 conn(0x7f12c803a240 legacy=0x7f12c803c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 10 monclient: shutdown 2024-05-08T15:55:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 >> v1:172.21.15.7:6792/0 conn(0x7f12e80a41b0 legacy=0x7f12e806e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 shutdown_connections 2024-05-08T15:55:39.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 >> 172.21.15.7:0/3054280281 conn(0x7f12e80a9fa0 msgr2=0x7f12e8130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 shutdown_connections 2024-05-08T15:55:39.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.059+0000 7f12ed11c640 1 -- 172.21.15.7:0/3054280281 wait complete. 2024-05-08T15:55:39.066 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:55:39.066 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon a 2024-05-08T15:55:39.066 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T15:55:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.174+0000 7ff83507b640 10 monclient: get_monmap_and_config 2024-05-08T15:55:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.174+0000 7ff83507b640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.174+0000 7ff83507b640 10 monclient: monmap: 2024-05-08T15:55:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:39.175496+0000 2024-05-08T15:55:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:39.175496+0000 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.171 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 1 Processor -- start 2024-05-08T15:55:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 1 -- start start 2024-05-08T15:55:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient: init 2024-05-08T15:55:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient: _add_conns ranks=[6,1,3,0,8,2,5,7,4] 2024-05-08T15:55:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient(hunting): picked mon.noname-d con 0x7ff830137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient(hunting): picked mon.noname-f con 0x7ff8300a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient(hunting): picked mon.noname-g con 0x7ff8300a41b0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8300a2c70 con 0x7ff8300a49a0 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8300a2f50 con 0x7ff8300a41b0 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8300a1670 con 0x7ff830137940 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff83507b640 10 monclient(hunting): authenticate will time out at 1347.290454s 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff82ed9d640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7ff830137940 0x7ff830137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35156/0 (socket says 172.21.15.7:35156) 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.175+0000 7ff82ed9d640 1 -- 172.21.15.7:0/2824461960 learned_addr learned my addr 172.21.15.7:0/2824461960 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3991184439 0 0) 0x7ff8300a1670 con 0x7ff830137940 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): my global_id is 124566 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff81c003520 con 0x7ff830137940 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536589041 0 0) 0x7ff8300a2f50 con 0x7ff8300a41b0 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): my global_id is 76881 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8300a1670 con 0x7ff8300a41b0 2024-05-08T15:55:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3301546121 0 0) 0x7ff8300a2c70 con 0x7ff8300a49a0 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient(hunting): my global_id is 6409 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8300a2f50 con 0x7ff8300a49a0 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 821855292 0 0) 0x7ff81c003520 con 0x7ff830137940 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 >> v1:172.21.15.81:6790/0 conn(0x7ff8300a41b0 legacy=0x7ff830140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 >> v1:172.21.15.81:6789/0 conn(0x7ff8300a49a0 legacy=0x7ff83013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 monclient: found mon.noname-d 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8300a39f0 con 0x7ff830137940 2024-05-08T15:55:39.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff8200017b0 con 0x7ff830137940 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff83507b640 5 monclient: authenticate success, global_id 124566 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: dump: 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.185 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff83507b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff820001a10 con 0x7ff830137940 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff83507b640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 1 -- 172.21.15.7:0/2824461960 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff820004be0 con 0x7ff830137940 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff83507b640 4 set_mon_vals no callback set 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff82e59c640 10 monclient: dump: 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.188 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff83507b640 10 monclient: shutdown 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff83507b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.176+0000 7ff83507b640 1 -- 172.21.15.7:0/2824461960 >> v1:172.21.15.7:6792/0 conn(0x7ff830137940 legacy=0x7ff830137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.177+0000 7ff83507b640 1 -- 172.21.15.7:0/2824461960 shutdown_connections 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.177+0000 7ff83507b640 1 -- 172.21.15.7:0/2824461960 >> 172.21.15.7:0/2824461960 conn(0x7ff8300a9fa0 msgr2=0x7ff8300aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.177+0000 7ff83507b640 1 -- 172.21.15.7:0/2824461960 shutdown_connections 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.177+0000 7ff83507b640 1 -- 172.21.15.7:0/2824461960 wait complete. 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.177+0000 7ff83507b640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.177+0000 7ff83507b640 10 monclient: monmap: 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:39.189 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.190 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 1 Processor -- start 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 1 -- start start 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient: init 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient: _add_conns ranks=[1,7,4,6,2,8,0,3,5] 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient(hunting): picked mon.noname-f con 0x7ff8300a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient(hunting): picked mon.noname-i con 0x7ff8300a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient(hunting): picked mon.noname-c con 0x7ff830137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8301340d0 con 0x7ff8300a41b0 2024-05-08T15:55:39.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff830135c40 con 0x7ff830137940 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff830135f20 con 0x7ff8300a49a0 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff83507b640 10 monclient(hunting): authenticate will time out at 1347.293511s 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff82f59e640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7ff830137940 0x7ff8301d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51622/0 (socket says 172.21.15.7:51622) 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.178+0000 7ff82f59e640 1 -- 172.21.15.7:0/3822055156 learned_addr learned my addr 172.21.15.7:0/3822055156 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151769441 0 0) 0x7ff830135c40 con 0x7ff830137940 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): my global_id is 124447 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff810003520 con 0x7ff830137940 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1011328574 0 0) 0x7ff830135f20 con 0x7ff8300a49a0 2024-05-08T15:55:39.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): my global_id is 115198 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff830135c40 con 0x7ff8300a49a0 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 136715706 0 0) 0x7ff8301340d0 con 0x7ff8300a41b0 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): my global_id is 6418 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff830135f20 con 0x7ff8300a41b0 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2202251042 0 0) 0x7ff810003520 con 0x7ff830137940 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8301340d0 con 0x7ff830137940 2024-05-08T15:55:39.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff828001d90 con 0x7ff830137940 2024-05-08T15:55:39.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:55:39.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): dump: 2024-05-08T15:55:39.194 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.194 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.195 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4123301256 0 0) 0x7ff8301340d0 con 0x7ff830137940 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.81:6792/0 conn(0x7ff8300a49a0 legacy=0x7ff830132750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.81:6789/0 conn(0x7ff8300a41b0 legacy=0x7ff830132040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 monclient: found mon.c 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff830135150 con 0x7ff830137940 2024-05-08T15:55:39.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff83507b640 5 monclient: authenticate success, global_id 124447 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff83507b640 10 monclient: _renew_subs 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff83507b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7ff8300a1f30 con 0x7ff830137940 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff83507b640 10 monclient: _renew_subs 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff83507b640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7ff830135960 con 0x7ff830137940 2024-05-08T15:55:39.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7ff828002090 con 0x7ff830137940 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff828003f80 con 0x7ff830137940 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff82dd9b640 4 set_mon_vals no callback set 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.179+0000 7ff827fff640 10 monclient: dump: 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.198 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.180+0000 7ff83507b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.180+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff8140065c0 con 0x7ff814002860 2024-05-08T15:55:39.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.180+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7ff828004120 con 0x7ff830137940 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.180+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7ff828046f80 con 0x7ff830137940 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869136636 0 0) 0x7ff8140065c0 con 0x7ff814002860 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient(hunting): my global_id is 95823 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8301340d0 con 0x7ff814002860 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff82659c640 10 monclient: get_auth_request con 0x7ff810039b00 auth_method 0 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 377639540 0 0) 0x7ff8301340d0 con 0x7ff814002860 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff8140065c0 con 0x7ff814002860 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff820001790 con 0x7ff814002860 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:39.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient: dump: 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.201 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2091615074 0 0) 0x7ff8140065c0 con 0x7ff814002860 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff81400a150 con 0x7ff814002860 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7ff820007480 con 0x7ff814002860 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.7:6789/0 conn(0x7ff814002860 legacy=0x7ff814004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.181+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.7:6789/0 conn(0x7ff814002860 legacy=0x7ff814004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff83507b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:39.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7ff814009e40 con 0x7ff814009100 2024-05-08T15:55:39.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573281738 0 0) 0x7ff814009e40 con 0x7ff814009100 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 10 monclient(hunting): my global_id is 95832 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff8140065c0 con 0x7ff814009100 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3732452621 0 0) 0x7ff8140065c0 con 0x7ff814009100 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7ff814009e40 con 0x7ff814009100 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7ff820005600 con 0x7ff814009100 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 10 monclient: dump: 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.205 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2527715055 0 0) 0x7ff814009e40 con 0x7ff814009100 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.188+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 --> v1:172.21.15.7:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff81400c3d0 con 0x7ff814009100 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 <== mon.0 v1:172.21.15.7:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7863 (unknown 0 0 2394474437) 0x7ff820007630 con 0x7ff814009100 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff827fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff827fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.7:6789/0 conn(0x7ff814009100 legacy=0x7ff814009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff827fff640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.7:6789/0 conn(0x7ff814009100 legacy=0x7ff814009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:39.207 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 389739, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.208 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.209 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.210 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.211 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.212 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:39.213 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.214 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.215 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.216 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.217 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:39.218 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.219 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.220 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.221 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:39.222 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.223 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.224 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.225 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.226 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": [ 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:39.227 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.81:6812/847627118 conn(0x7ff810039b00 legacy=0x7ff81003bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 10 monclient: shutdown 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 >> v1:172.21.15.7:6791/0 conn(0x7ff830137940 legacy=0x7ff8301d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff82f59e640 1 -- 172.21.15.7:0/3822055156 reap_dead start 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff82f59e640 1 -- 172.21.15.7:0/3822055156 reap_dead start 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 shutdown_connections 2024-05-08T15:55:39.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 >> 172.21.15.7:0/3822055156 conn(0x7ff8300a9fa0 msgr2=0x7ff8301308f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 shutdown_connections 2024-05-08T15:55:39.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.189+0000 7ff83507b640 1 -- 172.21.15.7:0/3822055156 wait complete. 2024-05-08T15:55:39.230 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:39.230 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon b 2024-05-08T15:55:39.230 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T15:55:39.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.338+0000 7fad6bd59640 10 monclient: get_monmap_and_config 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.338+0000 7fad6bd59640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.338+0000 7fad6bd59640 10 monclient: monmap: 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:39.339097+0000 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:39.339097+0000 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.334 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.335 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.338+0000 7fad6bd59640 1 Processor -- start 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.338+0000 7fad6bd59640 1 -- start start 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.338+0000 7fad6bd59640 10 monclient: init 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 10 monclient: _add_conns ranks=[1,5,2,7,6,4,8,3,0] 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 10 monclient(hunting): picked mon.noname-f con 0x7fad640a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 10 monclient(hunting): picked mon.noname-h con 0x7fad640a5780 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 10 monclient(hunting): picked mon.noname-b con 0x7fad6413e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad640a1660 con 0x7fad640a53a0 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad640a28f0 con 0x7fad6413e310 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad640a2bd0 con 0x7fad640a5780 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6bd59640 10 monclient(hunting): authenticate will time out at 1347.454169s 2024-05-08T15:55:39.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6a2f6640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fad6413e310 0x7fad64140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51034/0 (socket says 172.21.15.7:51034) 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad6a2f6640 1 -- 172.21.15.7:0/3155169325 learned_addr learned my addr 172.21.15.7:0/3155169325 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094804316 0 0) 0x7fad640a28f0 con 0x7fad6413e310 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad68af3640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad68af3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad68af3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad68af3640 10 monclient(hunting): my global_id is 96041 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.339+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad4c003520 con 0x7fad6413e310 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2341568902 0 0) 0x7fad640a1660 con 0x7fad640a53a0 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient(hunting): my global_id is 6427 2024-05-08T15:55:39.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad640a28f0 con 0x7fad640a53a0 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 245673841 0 0) 0x7fad4c003520 con 0x7fad6413e310 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 >> v1:172.21.15.81:6791/0 conn(0x7fad640a5780 legacy=0x7fad6413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 >> v1:172.21.15.81:6789/0 conn(0x7fad640a53a0 legacy=0x7fad640a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 monclient: found mon.noname-b 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad640a6730 con 0x7fad6413e310 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fad54001ab0 con 0x7fad6413e310 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 5 monclient: authenticate success, global_id 96041 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: dump: 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.347 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fad54001d10 con 0x7fad6413e310 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:39.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 1 -- 172.21.15.7:0/3155169325 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fad54004f60 con 0x7fad6413e310 2024-05-08T15:55:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 4 set_mon_vals no callback set 2024-05-08T15:55:39.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad68af3640 10 monclient: dump: 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.350 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 10 monclient: shutdown 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 1 -- 172.21.15.7:0/3155169325 >> v1:172.21.15.7:6790/0 conn(0x7fad6413e310 legacy=0x7fad64140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 1 -- 172.21.15.7:0/3155169325 shutdown_connections 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 1 -- 172.21.15.7:0/3155169325 >> 172.21.15.7:0/3155169325 conn(0x7fad640a9fa0 msgr2=0x7fad640aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 1 -- 172.21.15.7:0/3155169325 shutdown_connections 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.340+0000 7fad6bd59640 1 -- 172.21.15.7:0/3155169325 wait complete. 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.341+0000 7fad6bd59640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.341+0000 7fad6bd59640 10 monclient: monmap: 2024-05-08T15:55:39.351 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.352 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.341+0000 7fad6bd59640 1 Processor -- start 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.341+0000 7fad6bd59640 1 -- start start 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.341+0000 7fad6bd59640 10 monclient: init 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 10 monclient: _add_conns ranks=[6,1,7,5,8,0,3,4,2] 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 10 monclient(hunting): picked mon.noname-d con 0x7fad640a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 10 monclient(hunting): picked mon.noname-f con 0x7fad640a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 10 monclient(hunting): picked mon.noname-i con 0x7fad6413e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:39.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad64134d70 con 0x7fad640a5780 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad640a1e00 con 0x7fad640a53a0 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad64134a20 con 0x7fad6413e310 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad6bd59640 10 monclient(hunting): authenticate will time out at 1347.457107s 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad69af5640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fad640a53a0 0x7fad6413bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35170/0 (socket says 172.21.15.7:35170) 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad69af5640 1 -- 172.21.15.7:0/1398581548 learned_addr learned my addr 172.21.15.7:0/1398581548 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2047741768 0 0) 0x7fad640a28f0 con 0x7fad640a53a0 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): my global_id is 124575 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad50003520 con 0x7fad640a53a0 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2251534955 0 0) 0x7fad64134d70 con 0x7fad640a5780 2024-05-08T15:55:39.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): my global_id is 6436 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad640a28f0 con 0x7fad640a5780 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1273924665 0 0) 0x7fad64134a20 con 0x7fad6413e310 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): my global_id is 115207 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad64134d70 con 0x7fad6413e310 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2672495535 0 0) 0x7fad50003520 con 0x7fad640a53a0 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fad64134a20 con 0x7fad640a53a0 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fad5c001b10 con 0x7fad640a53a0 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): dump: 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.356 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.342+0000 7fad627fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3821644619 0 0) 0x7fad640a28f0 con 0x7fad640a5780 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fad50003520 con 0x7fad640a5780 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fad58001980 con 0x7fad640a5780 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient(hunting): dump: 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.357 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.358 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2553405650 0 0) 0x7fad64134a20 con 0x7fad640a53a0 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.81:6792/0 conn(0x7fad6413e310 legacy=0x7fad641d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.81:6789/0 conn(0x7fad640a5780 legacy=0x7fad6413c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 monclient: found mon.d 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad640a43c0 con 0x7fad640a53a0 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad6bd59640 5 monclient: authenticate success, global_id 124575 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad6bd59640 10 monclient: _renew_subs 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad6bd59640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fad6413cb70 con 0x7fad640a53a0 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad6bd59640 10 monclient: _renew_subs 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad6bd59640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7fad64135200 con 0x7fad640a53a0 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fad5c0038c0 con 0x7fad640a53a0 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fad5c003db0 con 0x7fad640a53a0 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad637fe640 4 set_mon_vals no callback set 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:39.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.343+0000 7fad627fc640 10 monclient: dump: 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.361 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fad5c003f50 con 0x7fad640a53a0 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad6bd59640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fad440065c0 con 0x7fad44002860 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fad5c046c40 con 0x7fad640a53a0 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3314723555 0 0) 0x7fad440065c0 con 0x7fad44002860 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 10 monclient(hunting): my global_id is 96050 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad64134a20 con 0x7fad44002860 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad6a2f6640 10 monclient: get_auth_request con 0x7fad50039f20 auth_method 0 2024-05-08T15:55:39.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3866704732 0 0) 0x7fad64134a20 con 0x7fad44002860 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fad440065c0 con 0x7fad44002860 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fad58003950 con 0x7fad44002860 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 10 monclient: dump: 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.363 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2823218227 0 0) 0x7fad440065c0 con 0x7fad44002860 2024-05-08T15:55:39.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.344+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad4400a150 con 0x7fad44002860 2024-05-08T15:55:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.345+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7fad58003d00 con 0x7fad44002860 2024-05-08T15:55:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.345+0000 7fad627fc640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.345+0000 7fad627fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.345+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.7:6790/0 conn(0x7fad44002860 legacy=0x7fad44004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.345+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.7:6790/0 conn(0x7fad44002860 legacy=0x7fad44004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad6bd59640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7fad44009e40 con 0x7fad44009100 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2689301261 0 0) 0x7fad44009e40 con 0x7fad44009100 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad627fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad627fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad627fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad627fc640 10 monclient(hunting): my global_id is 96059 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad440065c0 con 0x7fad44009100 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.351+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3493668275 0 0) 0x7fad440065c0 con 0x7fad44009100 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fad44009e40 con 0x7fad44009100 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fad580044e0 con 0x7fad44009100 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:39.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 10 monclient: dump: 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.368 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3280639466 0 0) 0x7fad44009e40 con 0x7fad44009100 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 --> v1:172.21.15.7:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fad4400c3d0 con 0x7fad44009100 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 <== mon.2 v1:172.21.15.7:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 4144635607) 0x7fad58005910 con 0x7fad44009100 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.7:6790/0 conn(0x7fad44009100 legacy=0x7fad44009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad627fc640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.7:6790/0 conn(0x7fad44009100 legacy=0x7fad44009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:39.369 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 389903, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:39.370 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.371 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.372 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.373 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.374 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:39.375 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.376 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.377 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.378 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.379 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.380 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.381 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:39.382 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.383 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:39.384 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.385 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.386 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.387 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.388 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:39.389 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:39.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.81:6812/847627118 conn(0x7fad50039f20 legacy=0x7fad5003c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 10 monclient: shutdown 2024-05-08T15:55:39.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 >> v1:172.21.15.7:6792/0 conn(0x7fad640a53a0 legacy=0x7fad6413bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6a2f6640 1 -- 172.21.15.7:0/1398581548 reap_dead start 2024-05-08T15:55:39.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6a2f6640 1 -- 172.21.15.7:0/1398581548 reap_dead start 2024-05-08T15:55:39.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 shutdown_connections 2024-05-08T15:55:39.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 >> 172.21.15.7:0/1398581548 conn(0x7fad640a9fa0 msgr2=0x7fad640a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 shutdown_connections 2024-05-08T15:55:39.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.352+0000 7fad6bd59640 1 -- 172.21.15.7:0/1398581548 wait complete. 2024-05-08T15:55:39.392 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:39.392 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon c 2024-05-08T15:55:39.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T15:55:39.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.501+0000 7f578869a640 10 monclient: get_monmap_and_config 2024-05-08T15:55:39.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.501+0000 7f578869a640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.501+0000 7f578869a640 10 monclient: monmap: 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:39.502039+0000 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:39.502039+0000 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.497 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.498 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.501+0000 7f578869a640 1 Processor -- start 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.501+0000 7f578869a640 1 -- start start 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.501+0000 7f578869a640 10 monclient: init 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 10 monclient: _add_conns ranks=[1,6,8,4,0,2,7,5,3] 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 10 monclient(hunting): picked mon.noname-f con 0x7f57800a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 10 monclient(hunting): picked mon.noname-d con 0x7f57800a5780 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 10 monclient(hunting): picked mon.noname-e con 0x7f578013e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800a1660 con 0x7f57800a53a0 2024-05-08T15:55:39.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800a28f0 con 0x7f57800a5780 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800a2bd0 con 0x7f578013e310 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f578869a640 10 monclient(hunting): authenticate will time out at 1347.616963s 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785c35640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f57800a5780 0x7f578013cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35176/0 (socket says 172.21.15.7:35176) 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785c35640 1 -- 172.21.15.7:0/361890728 learned_addr learned my addr 172.21.15.7:0/361890728 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5786c37640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f578013e310 0x7f5780140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55832/0 (socket says 172.21.15.7:55832) 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3817620091 0 0) 0x7f57800a2bd0 con 0x7f578013e310 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): my global_id is 144116 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5768003520 con 0x7f578013e310 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1056889692 0 0) 0x7f57800a28f0 con 0x7f57800a5780 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): my global_id is 124584 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57800a2bd0 con 0x7f57800a5780 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926576240 0 0) 0x7f57800a1660 con 0x7f57800a53a0 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 10 monclient(hunting): my global_id is 6445 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57800a28f0 con 0x7f57800a53a0 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.502+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3348943031 0 0) 0x7f5768003520 con 0x7f578013e310 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 >> v1:172.21.15.7:6792/0 conn(0x7f57800a5780 legacy=0x7f578013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 >> v1:172.21.15.81:6789/0 conn(0x7f57800a53a0 legacy=0x7f57800a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 1 monclient: found mon.noname-e 2024-05-08T15:55:39.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57800a6730 con 0x7f578013e310 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 5 monclient: authenticate success, global_id 144116 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f57700017b0 con 0x7f578013e310 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: dump: 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.509 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5770001ad0 con 0x7f578013e310 2024-05-08T15:55:39.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 1 -- 172.21.15.7:0/361890728 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5770004b20 con 0x7f578013e310 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 4 set_mon_vals no callback set 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f5785434640 10 monclient: dump: 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.511 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 10 monclient: shutdown 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 1 -- 172.21.15.7:0/361890728 >> v1:172.21.15.7:6793/0 conn(0x7f578013e310 legacy=0x7f5780140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 1 -- 172.21.15.7:0/361890728 shutdown_connections 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 1 -- 172.21.15.7:0/361890728 >> 172.21.15.7:0/361890728 conn(0x7f57800a9fa0 msgr2=0x7f57800aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 1 -- 172.21.15.7:0/361890728 shutdown_connections 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.503+0000 7f578869a640 1 -- 172.21.15.7:0/361890728 wait complete. 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.504+0000 7f578869a640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.504+0000 7f578869a640 10 monclient: monmap: 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.513 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.504+0000 7f578869a640 1 Processor -- start 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.504+0000 7f578869a640 1 -- start start 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.504+0000 7f578869a640 10 monclient: init 2024-05-08T15:55:39.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 10 monclient: _add_conns ranks=[1,3,5,0,8,7,6,4,2] 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 10 monclient(hunting): picked mon.noname-f con 0x7f57800a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 10 monclient(hunting): picked mon.noname-g con 0x7f57800a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 10 monclient(hunting): picked mon.noname-h con 0x7f578013e310 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800a1e00 con 0x7f57800a53a0 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57800a43c0 con 0x7f57800a5780 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5780132e70 con 0x7f578013e310 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f578869a640 10 monclient(hunting): authenticate will time out at 1347.619879s 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f5786436640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f57800a53a0 0x7f578012e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:46282/0 (socket says 172.21.15.7:46282) 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f5786436640 1 -- 172.21.15.7:0/3475969634 learned_addr learned my addr 172.21.15.7:0/3475969634 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2090591137 0 0) 0x7f57800a43c0 con 0x7f57800a5780 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): my global_id is 76890 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f576c003520 con 0x7f57800a5780 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1588354733 0 0) 0x7f57800a1e00 con 0x7f57800a53a0 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): my global_id is 6454 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57800a43c0 con 0x7f57800a53a0 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2472858303 0 0) 0x7f5780132e70 con 0x7f578013e310 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): my global_id is 6656 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57800a1e00 con 0x7f578013e310 2024-05-08T15:55:39.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 826342197 0 0) 0x7f576c003520 con 0x7f57800a5780 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5780132e70 con 0x7f57800a5780 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5774001d50 con 0x7f57800a5780 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.505+0000 7f577effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): got monmap 1 from mon.noname-g (according to old e1) 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): dump: 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.517 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2512073073 0 0) 0x7f57800a43c0 con 0x7f57800a53a0 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f576c003520 con 0x7f57800a53a0 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5778001a90 con 0x7f57800a53a0 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): dump: 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.518 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 78895942 0 0) 0x7f57800a1e00 con 0x7f578013e310 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f57800a43c0 con 0x7f578013e310 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5770001770 con 0x7f578013e310 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient(hunting): dump: 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.520 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.521 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3892468884 0 0) 0x7f5780132e70 con 0x7f57800a5780 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.81:6791/0 conn(0x7f578013e310 legacy=0x7f57801d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.81:6789/0 conn(0x7f57800a53a0 legacy=0x7f578012e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 monclient: found mon.g 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5780134e80 con 0x7f57800a5780 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 5 monclient: authenticate success, global_id 76890 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 10 monclient: _renew_subs 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f57801348c0 con 0x7f57800a5780 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 10 monclient: _renew_subs 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 10 monclient: _send_mon_message to mon.g at v1:172.21.15.81:6790/0 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.81:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f5780138700 con 0x7f57800a5780 2024-05-08T15:55:39.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5774004b60 con 0x7f57800a5780 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5774005050 con 0x7f57800a5780 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f5784c33640 4 set_mon_vals no callback set 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f577effd640 10 monclient: dump: 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.524 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.506+0000 7f578869a640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f57600065c0 con 0x7f5760002860 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f57740052b0 con 0x7f57800a5780 2024-05-08T15:55:39.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.3 v1:172.21.15.81:6790/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f5774046ff0 con 0x7f57800a5780 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1187810124 0 0) 0x7f57600065c0 con 0x7f5760002860 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 10 monclient(hunting): my global_id is 124456 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5780132e70 con 0x7f5760002860 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f5786c37640 10 monclient: get_auth_request con 0x7f576c03a470 auth_method 0 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2032013346 0 0) 0x7f5780132e70 con 0x7f5760002860 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f57600065c0 con 0x7f5760002860 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5778003b80 con 0x7f5760002860 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.507+0000 7f577effd640 10 monclient: dump: 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.526 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.508+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1845186146 0 0) 0x7f57600065c0 con 0x7f5760002860 2024-05-08T15:55:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.508+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f576000a150 con 0x7f5760002860 2024-05-08T15:55:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.508+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f5778003df0 con 0x7f5760002860 2024-05-08T15:55:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.508+0000 7f577effd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.508+0000 7f577effd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.508+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.7:6791/0 conn(0x7f5760002860 legacy=0x7f5760004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.508+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.7:6791/0 conn(0x7f5760002860 legacy=0x7f5760004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.514+0000 7f578869a640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.514+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f5760009e20 con 0x7f57600090e0 2024-05-08T15:55:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2691699859 0 0) 0x7f5760009e20 con 0x7f57600090e0 2024-05-08T15:55:39.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient(hunting): my global_id is 124465 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57600065c0 con 0x7f57600090e0 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1481652405 0 0) 0x7f57600065c0 con 0x7f57600090e0 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5760009e20 con 0x7f57600090e0 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f57780057c0 con 0x7f57600090e0 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient: dump: 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.530 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1570904088 0 0) 0x7f5760009e20 con 0x7f57600090e0 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 --> v1:172.21.15.7:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f576000c390 con 0x7f57600090e0 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 <== mon.4 v1:172.21.15.7:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 3214660956) 0x7f5778007a80 con 0x7f57600090e0 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.7:6791/0 conn(0x7f57600090e0 legacy=0x7f5760009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.515+0000 7f577effd640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.7:6791/0 conn(0x7f57600090e0 legacy=0x7f5760009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:39.532 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 147685, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 0, 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.533 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:39.534 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:39.535 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.536 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.537 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.538 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.539 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.540 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.541 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.542 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.543 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.544 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.545 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.546 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.547 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.548 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.549 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:39.550 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.81:6812/847627118 conn(0x7f576c03a470 legacy=0x7f576c03c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 10 monclient: shutdown 2024-05-08T15:55:39.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 >> v1:172.21.15.81:6790/0 conn(0x7f57800a5780 legacy=0x7f57801d1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f5786c37640 1 -- 172.21.15.7:0/3475969634 reap_dead start 2024-05-08T15:55:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f5786c37640 1 -- 172.21.15.7:0/3475969634 reap_dead start 2024-05-08T15:55:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 shutdown_connections 2024-05-08T15:55:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 >> 172.21.15.7:0/3475969634 conn(0x7f57800a9fa0 msgr2=0x7f578013efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 shutdown_connections 2024-05-08T15:55:39.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.516+0000 7f578869a640 1 -- 172.21.15.7:0/3475969634 wait complete. 2024-05-08T15:55:39.552 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:39.552 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon d 2024-05-08T15:55:39.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.d mon_status 2024-05-08T15:55:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.660+0000 7f2d649fb640 10 monclient: get_monmap_and_config 2024-05-08T15:55:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.660+0000 7f2d649fb640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.661+0000 7f2d649fb640 10 monclient: monmap: 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:39.661962+0000 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:39.661962+0000 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.657 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.658 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.661+0000 7f2d649fb640 1 Processor -- start 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.661+0000 7f2d649fb640 1 -- start start 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.661+0000 7f2d649fb640 10 monclient: init 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 10 monclient: _add_conns ranks=[3,8,1,7,5,2,6,0,4] 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 10 monclient(hunting): picked mon.noname-g con 0x7f2d60137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 10 monclient(hunting): picked mon.noname-e con 0x7f2d600a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 10 monclient(hunting): picked mon.noname-f con 0x7f2d600a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:39.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d600a2c70 con 0x7f2d600a41b0 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d600a2f50 con 0x7f2d60137940 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d600a1670 con 0x7f2d600a49a0 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 10 monclient(hunting): authenticate will time out at 1347.776944s 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5dd9b640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f2d600a49a0 0x7f2d6013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55842/0 (socket says 172.21.15.7:55842) 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5dd9b640 1 -- 172.21.15.7:0/794140916 learned_addr learned my addr 172.21.15.7:0/794140916 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3588281249 0 0) 0x7f2d600a1670 con 0x7f2d600a49a0 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): my global_id is 144125 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d4c003520 con 0x7f2d600a49a0 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2646988093 0 0) 0x7f2d600a2c70 con 0x7f2d600a41b0 2024-05-08T15:55:39.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): my global_id is 6463 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d600a1670 con 0x7f2d600a41b0 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 45071998 0 0) 0x7f2d600a2f50 con 0x7f2d60137940 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient(hunting): my global_id is 76899 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d600a2c70 con 0x7f2d60137940 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1277913888 0 0) 0x7f2d4c003520 con 0x7f2d600a49a0 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 >> v1:172.21.15.81:6790/0 conn(0x7f2d60137940 legacy=0x7f2d60137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 >> v1:172.21.15.81:6789/0 conn(0x7f2d600a41b0 legacy=0x7f2d60140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 monclient: found mon.noname-e 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d600a39f0 con 0x7f2d600a49a0 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2d500017b0 con 0x7f2d600a49a0 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 5 monclient: authenticate success, global_id 144125 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d5d59a640 10 monclient: dump: 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.670 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.662+0000 7f2d649fb640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2d50001ad0 con 0x7f2d600a49a0 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d5d59a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d5d59a640 1 -- 172.21.15.7:0/794140916 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2d50004be0 con 0x7f2d600a49a0 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d5d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 4 set_mon_vals no callback set 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d5d59a640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d5d59a640 10 monclient: dump: 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.671 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.672 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 10 monclient: shutdown 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 1 -- 172.21.15.7:0/794140916 >> v1:172.21.15.7:6793/0 conn(0x7f2d600a49a0 legacy=0x7f2d6013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 1 -- 172.21.15.7:0/794140916 shutdown_connections 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 1 -- 172.21.15.7:0/794140916 >> 172.21.15.7:0/794140916 conn(0x7f2d600a9fa0 msgr2=0x7f2d600aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 1 -- 172.21.15.7:0/794140916 shutdown_connections 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.663+0000 7f2d649fb640 1 -- 172.21.15.7:0/794140916 wait complete. 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.664+0000 7f2d649fb640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.664+0000 7f2d649fb640 10 monclient: monmap: 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.674 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 1 Processor -- start 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 1 -- start start 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient: init 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient: _add_conns ranks=[0,7,5,6,1,2,8,3,4] 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient(hunting): picked mon.noname-a con 0x7f2d600a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient(hunting): picked mon.noname-i con 0x7f2d600a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient(hunting): picked mon.noname-h con 0x7f2d60137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d60133350 con 0x7f2d600a41b0 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d60135de0 con 0x7f2d60137940 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d60132c80 con 0x7f2d600a49a0 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d649fb640 10 monclient(hunting): authenticate will time out at 1347.780287s 2024-05-08T15:55:39.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d5e59c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f2d600a41b0 0x7f2d60131e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34006/0 (socket says 172.21.15.7:34006) 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d5e59c640 1 -- 172.21.15.7:0/887343611 learned_addr learned my addr 172.21.15.7:0/887343611 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393276565 0 0) 0x7f2d600a2c70 con 0x7f2d600a41b0 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d56ffd640 10 monclient(hunting): my global_id is 95841 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.665+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d40003520 con 0x7f2d600a41b0 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 983054938 0 0) 0x7f2d60135de0 con 0x7f2d60137940 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): my global_id is 6665 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d600a2c70 con 0x7f2d60137940 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2199036456 0 0) 0x7f2d60132c80 con 0x7f2d600a49a0 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): my global_id is 115216 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d60135de0 con 0x7f2d600a49a0 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2141482357 0 0) 0x7f2d40003520 con 0x7f2d600a41b0 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2d60132c80 con 0x7f2d600a41b0 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2d48001a30 con 0x7f2d600a41b0 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.678 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3553945313 0 0) 0x7f2d600a2c70 con 0x7f2d60137940 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2d40003520 con 0x7f2d60137940 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2d58001dd0 con 0x7f2d60137940 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.679 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 655796259 0 0) 0x7f2d60132c80 con 0x7f2d600a41b0 2024-05-08T15:55:39.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.81:6792/0 conn(0x7f2d600a49a0 legacy=0x7f2d60132570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.81:6791/0 conn(0x7f2d60137940 legacy=0x7f2d601d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 monclient: found mon.a 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d60133f90 con 0x7f2d600a41b0 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d649fb640 5 monclient: authenticate success, global_id 95841 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d649fb640 10 monclient: _renew_subs 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d649fb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f2d600a1f30 con 0x7f2d600a41b0 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d649fb640 10 monclient: _renew_subs 2024-05-08T15:55:39.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d649fb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f2d60135b00 con 0x7f2d600a41b0 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2d480038c0 con 0x7f2d600a41b0 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2d48003db0 con 0x7f2d600a41b0 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d5cd99640 4 set_mon_vals no callback set 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.666+0000 7f2d56ffd640 10 monclient: dump: 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.682 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f2d48003fd0 con 0x7f2d600a41b0 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d649fb640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2d440065c0 con 0x7f2d44002860 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f2d48046d80 con 0x7f2d600a41b0 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d5dd9b640 10 monclient: get_auth_request con 0x7f2d40039ff0 auth_method 0 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260251057 0 0) 0x7f2d440065c0 con 0x7f2d44002860 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 10 monclient(hunting): my global_id is 124593 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.667+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d60132c80 con 0x7f2d44002860 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3757013310 0 0) 0x7f2d60132c80 con 0x7f2d44002860 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2d440065c0 con 0x7f2d44002860 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2d58003cd0 con 0x7f2d44002860 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:39.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 10 monclient: dump: 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.685 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2686846127 0 0) 0x7f2d440065c0 con 0x7f2d44002860 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2d4400a150 con 0x7f2d44002860 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2d580051f0 con 0x7f2d44002860 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.7:6792/0 conn(0x7f2d44002860 legacy=0x7f2d44004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.668+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.7:6792/0 conn(0x7f2d44002860 legacy=0x7f2d44004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d649fb640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2d44009e40 con 0x7f2d44009100 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1067562067 0 0) 0x7f2d44009e40 con 0x7f2d44009100 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient(hunting): my global_id is 124602 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d440065c0 con 0x7f2d44009100 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2273102219 0 0) 0x7f2d440065c0 con 0x7f2d44009100 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2d44009e40 con 0x7f2d44009100 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2d58005950 con 0x7f2d44009100 2024-05-08T15:55:39.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient: dump: 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.689 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 4092232848 0 0) 0x7f2d44009e40 con 0x7f2d44009100 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 --> v1:172.21.15.7:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2d4400c3d0 con 0x7f2d44009100 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 <== mon.6 v1:172.21.15.7:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7523 (unknown 0 0 2660863472) 0x7f2d58005bd0 con 0x7f2d44009100 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.7:6792/0 conn(0x7f2d44009100 legacy=0x7f2d44009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.675+0000 7f2d56ffd640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.7:6792/0 conn(0x7f2d44009100 legacy=0x7f2d44009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 147845, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:39.691 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.692 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.693 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:39.694 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.695 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.696 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.697 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.698 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.699 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.700 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:39.701 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.702 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:39.703 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.704 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:39.705 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.706 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:39.707 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.708 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:39.709 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.81:6812/847627118 conn(0x7f2d40039ff0 legacy=0x7f2d4003c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 10 monclient: shutdown 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 >> v1:172.21.15.7:6789/0 conn(0x7f2d600a41b0 legacy=0x7f2d60131e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d5ed9d640 1 -- 172.21.15.7:0/887343611 reap_dead start 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d5ed9d640 1 -- 172.21.15.7:0/887343611 reap_dead start 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 shutdown_connections 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 >> 172.21.15.7:0/887343611 conn(0x7f2d600a9fa0 msgr2=0x7f2d600a8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 shutdown_connections 2024-05-08T15:55:39.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.676+0000 7f2d649fb640 1 -- 172.21.15.7:0/887343611 wait complete. 2024-05-08T15:55:39.712 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:39.712 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon e 2024-05-08T15:55:39.712 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.e mon_status 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.823+0000 7f42dab50640 10 monclient: get_monmap_and_config 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.823+0000 7f42dab50640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.823+0000 7f42dab50640 10 monclient: monmap: 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:39.824337+0000 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:39.824337+0000 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.819 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.820 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.820 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 1 Processor -- start 2024-05-08T15:55:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 1 -- start start 2024-05-08T15:55:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient: init 2024-05-08T15:55:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient: _add_conns ranks=[6,8,5,1,0,3,4,7,2] 2024-05-08T15:55:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient(hunting): picked mon.noname-d con 0x7f42d4137da0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:39.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient(hunting): picked mon.noname-e con 0x7f42d4137870 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient(hunting): picked mon.noname-h con 0x7f42d413e2b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d406c600 con 0x7f42d413e2b0 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d406dcf0 con 0x7f42d4137da0 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d406dfd0 con 0x7f42d4137870 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42dab50640 10 monclient(hunting): authenticate will time out at 1347.939290s 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42d88ec640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f42d4137da0 0x7f42d4138180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35206/0 (socket says 172.21.15.7:35206) 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.824+0000 7f42d88ec640 1 -- 172.21.15.7:0/380521674 learned_addr learned my addr 172.21.15.7:0/380521674 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1872278723 0 0) 0x7f42d406dcf0 con 0x7f42d4137da0 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): my global_id is 124611 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42c0003520 con 0x7f42d4137da0 2024-05-08T15:55:39.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963936722 0 0) 0x7f42d406dfd0 con 0x7f42d4137870 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient(hunting): my global_id is 144134 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d406dcf0 con 0x7f42d4137870 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4162776773 0 0) 0x7f42c0003520 con 0x7f42d4137da0 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 >> v1:172.21.15.7:6793/0 conn(0x7f42d4137870 legacy=0x7f42d413cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 >> v1:172.21.15.81:6791/0 conn(0x7f42d413e2b0 legacy=0x7f42d41406a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 monclient: found mon.noname-d 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d406e2b0 con 0x7f42d4137da0 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f42c4001b30 con 0x7f42d4137da0 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 5 monclient: authenticate success, global_id 124611 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: dump: 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.830 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f42c4001d90 con 0x7f42d4137da0 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 1 -- 172.21.15.7:0/380521674 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f42c4004f60 con 0x7f42d4137da0 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:39.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 4 set_mon_vals no callback set 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42d37fe640 10 monclient: dump: 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.832 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 10 monclient: shutdown 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 1 -- 172.21.15.7:0/380521674 >> v1:172.21.15.7:6792/0 conn(0x7f42d4137da0 legacy=0x7f42d4138180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 1 -- 172.21.15.7:0/380521674 shutdown_connections 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.825+0000 7f42dab50640 1 -- 172.21.15.7:0/380521674 >> 172.21.15.7:0/380521674 conn(0x7f42d40a8440 msgr2=0x7f42d40a4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.826+0000 7f42dab50640 1 -- 172.21.15.7:0/380521674 shutdown_connections 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.826+0000 7f42dab50640 1 -- 172.21.15.7:0/380521674 wait complete. 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.826+0000 7f42dab50640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.826+0000 7f42dab50640 10 monclient: monmap: 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:39.833 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.826+0000 7f42dab50640 1 Processor -- start 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 1 -- start start 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient: init 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient: _add_conns ranks=[7,8,5,3,2,6,4,1,0] 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient(hunting): picked mon.noname-i con 0x7f42d4137870 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient(hunting): picked mon.noname-e con 0x7f42d4137da0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient(hunting): picked mon.noname-h con 0x7f42d413e2b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d4139780 con 0x7f42d413e2b0 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d40729e0 con 0x7f42d4137870 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42d4072600 con 0x7f42d4137da0 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42dab50640 10 monclient(hunting): authenticate will time out at 1347.942309s 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d3fff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f42d4137da0 0x7f42d4139070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55852/0 (socket says 172.21.15.7:55852) 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d3fff640 1 -- 172.21.15.7:0/3379608313 learned_addr learned my addr 172.21.15.7:0/3379608313 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 827308529 0 0) 0x7f42d4072600 con 0x7f42d4137da0 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d17fa640 10 monclient(hunting): my global_id is 144143 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.827+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b4003520 con 0x7f42d4137da0 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1184318239 0 0) 0x7f42d4139780 con 0x7f42d413e2b0 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): my global_id is 6683 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d4072600 con 0x7f42d413e2b0 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2012085985 0 0) 0x7f42b4003520 con 0x7f42d4137da0 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42d4139780 con 0x7f42d4137da0 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897141467 0 0) 0x7f42d40729e0 con 0x7f42d4137870 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): my global_id is 115225 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b4003520 con 0x7f42d4137870 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f42bc0015b0 con 0x7f42d4137da0 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): dump: 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.837 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2265485156 0 0) 0x7f42d4139780 con 0x7f42d4137da0 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.81:6792/0 conn(0x7f42d4137870 legacy=0x7f42d4138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.81:6791/0 conn(0x7f42d413e2b0 legacy=0x7f42d41d4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 monclient: found mon.e 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d413b9b0 con 0x7f42d4137da0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42dab50640 5 monclient: authenticate success, global_id 144143 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42dab50640 10 monclient: _renew_subs 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42dab50640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f42d413a210 con 0x7f42d4137da0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42dab50640 10 monclient: _renew_subs 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42dab50640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f42d413a490 con 0x7f42d4137da0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f42bc0038a0 con 0x7f42d4137da0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f42bc003d90 con 0x7f42d4137da0 2024-05-08T15:55:39.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d17fa640 10 monclient: dump: 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.840 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.828+0000 7f42d2ffd640 4 set_mon_vals no callback set 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f42bc003fb0 con 0x7f42d4137da0 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42dab50640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f42b80065c0 con 0x7f42b8002860 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f42bc046d70 con 0x7f42d4137da0 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3754356355 0 0) 0x7f42b80065c0 con 0x7f42b8002860 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 10 monclient(hunting): my global_id is 144152 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42d4139780 con 0x7f42b8002860 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d90ed640 10 monclient: get_auth_request con 0x7f42b4039df0 auth_method 0 2024-05-08T15:55:39.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3017960628 0 0) 0x7f42d4139780 con 0x7f42b8002860 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42b80065c0 con 0x7f42b8002860 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f42c4001250 con 0x7f42b8002860 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 10 monclient: dump: 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.842 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.829+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1726500001 0 0) 0x7f42b80065c0 con 0x7f42b8002860 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.830+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42b800a150 con 0x7f42b8002860 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.830+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f42c4001560 con 0x7f42b8002860 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.830+0000 7f42d17fa640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.830+0000 7f42d17fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.830+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.7:6793/0 conn(0x7f42b8002860 legacy=0x7f42b8004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.830+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.7:6793/0 conn(0x7f42b8002860 legacy=0x7f42b8004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.836+0000 7f42dab50640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.836+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f42b8009e40 con 0x7f42b8009100 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253630498 0 0) 0x7f42b8009e40 con 0x7f42b8009100 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient(hunting): my global_id is 144161 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42b80065c0 con 0x7f42b8009100 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2530199543 0 0) 0x7f42b80065c0 con 0x7f42b8009100 2024-05-08T15:55:39.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f42b8009e40 con 0x7f42b8009100 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f42c4001da0 con 0x7f42b8009100 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient: dump: 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.846 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3034887575 0 0) 0x7f42b8009e40 con 0x7f42b8009100 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 --> v1:172.21.15.7:6793/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f42b800c3d0 con 0x7f42b8009100 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 <== mon.8 v1:172.21.15.7:6793/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7522 (unknown 0 0 683698286) 0x7f42c4008760 con 0x7f42b8009100 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.7:6793/0 conn(0x7f42b8009100 legacy=0x7f42b8009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.837+0000 7f42d17fa640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.7:6793/0 conn(0x7f42b8009100 legacy=0x7f42b8009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:39.847 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 16085, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:39.848 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.849 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.850 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:39.851 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:39.852 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.853 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:39.854 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:39.856 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:39.858 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:39.860 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.861 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:39.862 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.863 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.864 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:39.865 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:39.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.81:6812/847627118 conn(0x7f42b4039df0 legacy=0x7f42b403c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 10 monclient: shutdown 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 >> v1:172.21.15.7:6793/0 conn(0x7f42d4137da0 legacy=0x7f42d4139070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42d90ed640 1 -- 172.21.15.7:0/3379608313 reap_dead start 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42d90ed640 1 -- 172.21.15.7:0/3379608313 reap_dead start 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 shutdown_connections 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 >> 172.21.15.7:0/3379608313 conn(0x7f42d40a8440 msgr2=0x7f42d40a75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 shutdown_connections 2024-05-08T15:55:39.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.838+0000 7f42dab50640 1 -- 172.21.15.7:0/3379608313 wait complete. 2024-05-08T15:55:39.867 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:39.868 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon f 2024-05-08T15:55:39.868 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.f mon_status 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.976+0000 7efe116a8640 10 monclient: get_monmap_and_config 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.976+0000 7efe116a8640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.976+0000 7efe116a8640 10 monclient: monmap: 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:39.977499+0000 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:39.977499+0000 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.972 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.973 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 1 Processor -- start 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 1 -- start start 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient: init 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient: _add_conns ranks=[3,6,5,0,7,2,1,8,4] 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient(hunting): picked mon.noname-g con 0x7efe0c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient(hunting): picked mon.noname-d con 0x7efe0c0a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:39.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient(hunting): picked mon.noname-h con 0x7efe0c0a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c0a2c70 con 0x7efe0c137940 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c0a2f50 con 0x7efe0c0a41b0 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c0a1670 con 0x7efe0c0a49a0 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe116a8640 10 monclient(hunting): authenticate will time out at 1348.092384s 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe0a7fc640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7efe0c0a49a0 0x7efe0c13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35210/0 (socket says 172.21.15.7:35210) 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.977+0000 7efe0a7fc640 1 -- 172.21.15.7:0/2126836528 learned_addr learned my addr 172.21.15.7:0/2126836528 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725400362 0 0) 0x7efe0c0a1670 con 0x7efe0c0a49a0 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): my global_id is 124620 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdf4003520 con 0x7efe0c0a49a0 2024-05-08T15:55:39.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 70142613 0 0) 0x7efe0c0a2f50 con 0x7efe0c0a41b0 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient(hunting): my global_id is 6692 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe0c0a1670 con 0x7efe0c0a41b0 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1210797576 0 0) 0x7efdf4003520 con 0x7efe0c0a49a0 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 >> v1:172.21.15.81:6791/0 conn(0x7efe0c0a41b0 legacy=0x7efe0c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 >> v1:172.21.15.81:6790/0 conn(0x7efe0c137940 legacy=0x7efe0c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 monclient: found mon.noname-d 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c0a39f0 con 0x7efe0c0a49a0 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 5 monclient: authenticate success, global_id 124620 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efe000017b0 con 0x7efe0c0a49a0 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: dump: 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.985 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efe00001ad0 con 0x7efe0c0a49a0 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 1 -- 172.21.15.7:0/2126836528 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efe00001d40 con 0x7efe0c0a49a0 2024-05-08T15:55:39.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 4 set_mon_vals no callback set 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe09ffb640 10 monclient: dump: 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.987 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 10 monclient: shutdown 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.978+0000 7efe116a8640 1 -- 172.21.15.7:0/2126836528 >> v1:172.21.15.7:6792/0 conn(0x7efe0c0a49a0 legacy=0x7efe0c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.979+0000 7efe116a8640 1 -- 172.21.15.7:0/2126836528 shutdown_connections 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.979+0000 7efe116a8640 1 -- 172.21.15.7:0/2126836528 >> 172.21.15.7:0/2126836528 conn(0x7efe0c0a9fa0 msgr2=0x7efe0c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.979+0000 7efe116a8640 1 -- 172.21.15.7:0/2126836528 shutdown_connections 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.979+0000 7efe116a8640 1 -- 172.21.15.7:0/2126836528 wait complete. 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.979+0000 7efe116a8640 10 monclient: build_initial_monmap 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.979+0000 7efe116a8640 10 monclient: monmap: 2024-05-08T15:55:39.988 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:39.989 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 1 Processor -- start 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 1 -- start start 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient: init 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient: _add_conns ranks=[8,5,3,1,2,0,7,6,4] 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient(hunting): picked mon.noname-e con 0x7efe0c0a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient(hunting): picked mon.noname-h con 0x7efe0c0a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient(hunting): picked mon.noname-g con 0x7efe0c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c133780 con 0x7efe0c137940 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c137720 con 0x7efe0c0a49a0 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe0c1332e0 con 0x7efe0c0a41b0 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:39.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe116a8640 10 monclient(hunting): authenticate will time out at 1348.095221s 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe0affd640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7efe0c0a41b0 0x7efe0c137010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55882/0 (socket says 172.21.15.7:55882) 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efe0affd640 1 -- 172.21.15.7:0/3139564990 learned_addr learned my addr 172.21.15.7:0/3139564990 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3279721919 0 0) 0x7efe0c1332e0 con 0x7efe0c0a41b0 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efdf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efdf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efdf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efdf37fe640 10 monclient(hunting): my global_id is 144170 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.980+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdf8003520 con 0x7efe0c0a41b0 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197252539 0 0) 0x7efe0c137720 con 0x7efe0c0a49a0 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): my global_id is 6701 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe0c1332e0 con 0x7efe0c0a49a0 2024-05-08T15:55:39.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2869557050 0 0) 0x7efe0c133780 con 0x7efe0c137940 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): my global_id is 76917 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe0c137720 con 0x7efe0c137940 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1807034839 0 0) 0x7efdf8003520 con 0x7efe0c0a41b0 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efe0c133780 con 0x7efe0c0a41b0 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efdfc001a10 con 0x7efe0c0a41b0 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): dump: 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.992 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2295294379 0 0) 0x7efe0c133780 con 0x7efe0c0a41b0 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.81:6791/0 conn(0x7efe0c0a49a0 legacy=0x7efe0c1d15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.81:6790/0 conn(0x7efe0c137940 legacy=0x7efe0c1d4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: _finish_hunting 0 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 monclient: found mon.e 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe0c0a1f30 con 0x7efe0c0a41b0 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: _finish_auth 0 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: _check_auth_tickets 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe116a8640 5 monclient: authenticate success, global_id 144170 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe116a8640 10 monclient: _renew_subs 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe116a8640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7efe0c0a0f60 con 0x7efe0c0a41b0 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe116a8640 10 monclient: _renew_subs 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe116a8640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7efe0c134470 con 0x7efe0c0a41b0 2024-05-08T15:55:39.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7efdfc0038c0 con 0x7efe0c0a41b0 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efdfc003e30 con 0x7efe0c0a41b0 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efe097fa640 4 set_mon_vals no callback set 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.981+0000 7efdf37fe640 10 monclient: dump: 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.996 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efe116a8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efdec0065c0 con 0x7efdec002860 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7efdfc003fd0 con 0x7efe0c0a41b0 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7efdfc046b90 con 0x7efe0c0a41b0 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efe0b7fe640 10 monclient: get_auth_request con 0x7efdf8039870 auth_method 0 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2768718764 0 0) 0x7efdec0065c0 con 0x7efdec002860 2024-05-08T15:55:39.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 10 monclient(hunting): my global_id is 6472 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe0c133780 con 0x7efdec002860 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 2892914157 0 0) 0x7efe0c133780 con 0x7efdec002860 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.982+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efdec0065c0 con 0x7efdec002860 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efe00004d50 con 0x7efdec002860 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 10 monclient: dump: 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1104912487 0 0) 0x7efdec0065c0 con 0x7efdec002860 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdec00a150 con 0x7efdec002860 2024-05-08T15:55:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7efe00006310 con 0x7efdec002860 2024-05-08T15:55:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.81:6789/0 conn(0x7efdec002860 legacy=0x7efdec004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.983+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.81:6789/0 conn(0x7efdec002860 legacy=0x7efdec004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efe116a8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7efdec009e20 con 0x7efdec0090e0 2024-05-08T15:55:40.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 362932067 0 0) 0x7efdec009e20 con 0x7efdec0090e0 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efdf37fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efdf37fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efdf37fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efdf37fe640 10 monclient(hunting): my global_id is 6481 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.990+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdec0065c0 con 0x7efdec0090e0 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 477054761 0 0) 0x7efdec0065c0 con 0x7efdec0090e0 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7efdec009e20 con 0x7efdec0090e0 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7efe00005380 con 0x7efdec0090e0 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 10 monclient: dump: 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.002 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2649915704 0 0) 0x7efdec009e20 con 0x7efdec0090e0 2024-05-08T15:55:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 --> v1:172.21.15.81:6789/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efdec00c390 con 0x7efdec0090e0 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 <== mon.1 v1:172.21.15.81:6789/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7525 (unknown 0 0 3886831868) 0x7efe00005650 con 0x7efdec0090e0 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.81:6789/0 conn(0x7efdec0090e0 legacy=0x7efdec009570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.991+0000 7efdf37fe640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.81:6789/0 conn(0x7efdec0090e0 legacy=0x7efdec009570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "leader", 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:40.004 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 532307, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:40.005 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:40.006 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.007 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.008 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.009 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:40.010 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.011 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:40.012 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.013 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.014 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.015 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.016 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.017 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:40.018 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.019 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.020 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.021 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.022 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.022 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:55:40.022 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.022 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.022 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.022 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:40.022 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:40.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.81:6812/847627118 conn(0x7efdf8039870 legacy=0x7efdf803bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 10 monclient: shutdown 2024-05-08T15:55:40.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 >> v1:172.21.15.7:6793/0 conn(0x7efe0c0a41b0 legacy=0x7efe0c137010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe0b7fe640 1 -- 172.21.15.7:0/3139564990 reap_dead start 2024-05-08T15:55:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe0b7fe640 1 -- 172.21.15.7:0/3139564990 reap_dead start 2024-05-08T15:55:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 shutdown_connections 2024-05-08T15:55:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 >> 172.21.15.7:0/3139564990 conn(0x7efe0c0a9fa0 msgr2=0x7efe0c130860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 shutdown_connections 2024-05-08T15:55:40.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:39.992+0000 7efe116a8640 1 -- 172.21.15.7:0/3139564990 wait complete. 2024-05-08T15:55:40.024 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:40.024 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon g 2024-05-08T15:55:40.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.g mon_status 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.132+0000 7f64e767b640 10 monclient: get_monmap_and_config 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.132+0000 7f64e767b640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.132+0000 7f64e767b640 10 monclient: monmap: 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:40.133684+0000 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:40.133684+0000 2024-05-08T15:55:40.128 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.129 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.133+0000 7f64e767b640 1 Processor -- start 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.133+0000 7f64e767b640 1 -- start start 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.133+0000 7f64e767b640 10 monclient: init 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.133+0000 7f64e767b640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.133+0000 7f64e767b640 10 monclient: _add_conns ranks=[7,2,6,5,4,1,8,0,3] 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.133+0000 7f64e767b640 10 monclient(hunting): picked mon.noname-i con 0x7f64e0137940 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.133+0000 7f64e767b640 10 monclient(hunting): picked mon.noname-b con 0x7f64e00a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 10 monclient(hunting): picked mon.noname-d con 0x7f64e00a41b0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e00a2c70 con 0x7f64e00a49a0 2024-05-08T15:55:40.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e00a2f50 con 0x7f64e00a41b0 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e00a1670 con 0x7f64e0137940 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 10 monclient(hunting): authenticate will time out at 1348.248676s 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e4c16640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f64e00a49a0 0x7f64e013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51074/0 (socket says 172.21.15.7:51074) 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e5c18640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f64e00a41b0 0x7f64e0140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35222/0 (socket says 172.21.15.7:35222) 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e4c16640 1 -- 172.21.15.7:0/1725332881 learned_addr learned my addr 172.21.15.7:0/1725332881 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2175070147 0 0) 0x7f64e00a2c70 con 0x7f64e00a49a0 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): my global_id is 96068 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64cc003520 con 0x7f64e00a49a0 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2486065151 0 0) 0x7f64e00a2f50 con 0x7f64e00a41b0 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): my global_id is 124629 2024-05-08T15:55:40.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e00a2c70 con 0x7f64e00a41b0 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118863588 0 0) 0x7f64e00a1670 con 0x7f64e0137940 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient(hunting): my global_id is 115234 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e00a2f50 con 0x7f64e0137940 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 175734607 0 0) 0x7f64cc003520 con 0x7f64e00a49a0 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 >> v1:172.21.15.81:6792/0 conn(0x7f64e0137940 legacy=0x7f64e0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 >> v1:172.21.15.7:6792/0 conn(0x7f64e00a41b0 legacy=0x7f64e0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 monclient: found mon.noname-b 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e00a39f0 con 0x7f64e00a49a0 2024-05-08T15:55:40.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64d00017b0 con 0x7f64e00a49a0 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 5 monclient: authenticate success, global_id 96068 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: dump: 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.144 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:40.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f64d0001a10 con 0x7f64e00a49a0 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 1 -- 172.21.15.7:0/1725332881 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64d0004be0 con 0x7f64e00a49a0 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64d7fff640 10 monclient: dump: 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.146 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.134+0000 7f64e767b640 4 set_mon_vals no callback set 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.135+0000 7f64e767b640 10 monclient: shutdown 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.135+0000 7f64e767b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.135+0000 7f64e767b640 1 -- 172.21.15.7:0/1725332881 >> v1:172.21.15.7:6790/0 conn(0x7f64e00a49a0 legacy=0x7f64e013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.135+0000 7f64e767b640 1 -- 172.21.15.7:0/1725332881 shutdown_connections 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.135+0000 7f64e767b640 1 -- 172.21.15.7:0/1725332881 >> 172.21.15.7:0/1725332881 conn(0x7f64e00a9fa0 msgr2=0x7f64e00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.135+0000 7f64e767b640 1 -- 172.21.15.7:0/1725332881 shutdown_connections 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.135+0000 7f64e767b640 1 -- 172.21.15.7:0/1725332881 wait complete. 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient: monmap: 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.148 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 1 Processor -- start 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 1 -- start start 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient: init 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient: _add_conns ranks=[7,3,0,2,4,1,8,6,5] 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient(hunting): picked mon.noname-i con 0x7f64e00a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient(hunting): picked mon.noname-g con 0x7f64e00a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient(hunting): picked mon.noname-a con 0x7f64e0137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e0135f30 con 0x7f64e0137940 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e0133f90 con 0x7f64e00a49a0 2024-05-08T15:55:40.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64e00a1f30 con 0x7f64e00a41b0 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.136+0000 7f64e767b640 10 monclient(hunting): authenticate will time out at 1348.251555s 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64e5c18640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f64e0137940 0x7f64e01d4c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34010/0 (socket says 172.21.15.7:34010) 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64e5c18640 1 -- 172.21.15.7:0/1364168263 learned_addr learned my addr 172.21.15.7:0/1364168263 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731926881 0 0) 0x7f64e0135f30 con 0x7f64e0137940 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): my global_id is 95850 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64c4003520 con 0x7f64e0137940 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515012053 0 0) 0x7f64e0133f90 con 0x7f64e00a49a0 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): my global_id is 76926 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e0135f30 con 0x7f64e00a49a0 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2313445425 0 0) 0x7f64e00a1f30 con 0x7f64e00a41b0 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): my global_id is 115243 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e0133f90 con 0x7f64e00a41b0 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 153238276 0 0) 0x7f64c4003520 con 0x7f64e0137940 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64e00a1f30 con 0x7f64e0137940 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64d80018d0 con 0x7f64e0137940 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:40.151 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.152 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1840840202 0 0) 0x7f64e0135f30 con 0x7f64e00a49a0 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64c4003520 con 0x7f64e00a49a0 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64d0004b20 con 0x7f64e00a49a0 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.153 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3654406654 0 0) 0x7f64e0133f90 con 0x7f64e00a41b0 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64e0135f30 con 0x7f64e00a41b0 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64dc001930 con 0x7f64e00a41b0 2024-05-08T15:55:40.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.155 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 588346207 0 0) 0x7f64e00a1f30 con 0x7f64e0137940 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.81:6792/0 conn(0x7f64e00a41b0 legacy=0x7f64e0131f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.81:6790/0 conn(0x7f64e00a49a0 legacy=0x7f64e0132630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 monclient: found mon.a 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e00a0f60 con 0x7f64e0137940 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64e767b640 5 monclient: authenticate success, global_id 95850 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64e767b640 10 monclient: _renew_subs 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64e767b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f64e0133960 con 0x7f64e0137940 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f64d8004e10 con 0x7f64e0137940 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64d8005300 con 0x7f64e0137940 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d5ffb640 10 monclient: dump: 2024-05-08T15:55:40.157 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.158 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.137+0000 7f64d77fe640 4 set_mon_vals no callback set 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.138+0000 7f64e767b640 10 monclient: _renew_subs 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.138+0000 7f64e767b640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.138+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f64e0134600 con 0x7f64e0137940 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.138+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f64d8010680 con 0x7f64e0137940 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.138+0000 7f64e767b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.138+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f64e00753d0 con 0x7f64e01372b0 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.138+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f64d80471a0 con 0x7f64e0137940 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64e4c16640 10 monclient: get_auth_request con 0x7f64c403a4a0 auth_method 0 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1749928385 0 0) 0x7f64e00753d0 con 0x7f64e01372b0 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient(hunting): my global_id is 76935 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e00a1f30 con 0x7f64e01372b0 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 22705600 0 0) 0x7f64e00a1f30 con 0x7f64e01372b0 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64e00753d0 con 0x7f64e01372b0 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64dc003930 con 0x7f64e01372b0 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient: dump: 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.160 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2271995395 0 0) 0x7f64e00753d0 con 0x7f64e01372b0 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f64e0137690 con 0x7f64e01372b0 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f64dc003a90 con 0x7f64e01372b0 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:40.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.81:6790/0 conn(0x7f64e01372b0 legacy=0x7f64e009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.139+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.81:6790/0 conn(0x7f64e01372b0 legacy=0x7f64e009ba50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64e767b640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f64e00008d0 con 0x7f64e002d050 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3022647985 0 0) 0x7f64e00008d0 con 0x7f64e002d050 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64d5ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64d5ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64d5ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64d5ffb640 10 monclient(hunting): my global_id is 76944 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.146+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e00753d0 con 0x7f64e002d050 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 731174379 0 0) 0x7f64e00753d0 con 0x7f64e002d050 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f64e00008d0 con 0x7f64e002d050 2024-05-08T15:55:40.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f64dc0042f0 con 0x7f64e002d050 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 10 monclient: got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 10 monclient: dump: 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.165 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 2132759595 0 0) 0x7f64e00008d0 con 0x7f64e002d050 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 --> v1:172.21.15.81:6790/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64e0132d40 con 0x7f64e002d050 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 <== mon.3 v1:172.21.15.81:6790/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 1723761987) 0x7f64dc0066f0 con 0x7f64e002d050 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.81:6790/0 conn(0x7f64e002d050 legacy=0x7f64e009c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.147+0000 7f64d5ffb640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.81:6790/0 conn(0x7f64e002d050 legacy=0x7f64e009c160 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 532437, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:40.167 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.168 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.169 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:40.170 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.171 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.172 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.173 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.174 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.175 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.176 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:40.177 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.178 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:40.179 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.180 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.181 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.182 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.183 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 3 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.184 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.185 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:40.185 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.81:6812/847627118 conn(0x7f64c403a4a0 legacy=0x7f64c403c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 10 monclient: shutdown 2024-05-08T15:55:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 >> v1:172.21.15.7:6789/0 conn(0x7f64e0137940 legacy=0x7f64e01d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e5c18640 1 -- 172.21.15.7:0/1364168263 reap_dead start 2024-05-08T15:55:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e5c18640 1 -- 172.21.15.7:0/1364168263 reap_dead start 2024-05-08T15:55:40.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 shutdown_connections 2024-05-08T15:55:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 >> 172.21.15.7:0/1364168263 conn(0x7f64e00a9fa0 msgr2=0x7f64e0130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 shutdown_connections 2024-05-08T15:55:40.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.148+0000 7f64e767b640 1 -- 172.21.15.7:0/1364168263 wait complete. 2024-05-08T15:55:40.187 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:40.187 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon h 2024-05-08T15:55:40.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.h mon_status 2024-05-08T15:55:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.295+0000 7f26a5a8f640 10 monclient: get_monmap_and_config 2024-05-08T15:55:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.295+0000 7f26a5a8f640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.295+0000 7f26a5a8f640 10 monclient: monmap: 2024-05-08T15:55:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:40.296734+0000 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:40.296734+0000 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.292 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.296+0000 7f26a5a8f640 1 Processor -- start 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.296+0000 7f26a5a8f640 1 -- start start 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.296+0000 7f26a5a8f640 10 monclient: init 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.296+0000 7f26a5a8f640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.296+0000 7f26a5a8f640 10 monclient: _add_conns ranks=[1,6,2,5,7,0,3,8,4] 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 10 monclient(hunting): picked mon.noname-f con 0x7f26a0137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 10 monclient(hunting): picked mon.noname-d con 0x7f26a00a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:40.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 10 monclient(hunting): picked mon.noname-b con 0x7f26a00a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a00a2c70 con 0x7f26a0137940 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a00a2f50 con 0x7f26a00a41b0 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a00a1670 con 0x7f26a00a49a0 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 10 monclient(hunting): authenticate will time out at 1348.411693s 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269effd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f26a00a49a0 0x7f26a013cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35230/0 (socket says 172.21.15.7:35230) 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269effd640 1 -- 172.21.15.7:0/1037338569 learned_addr learned my addr 172.21.15.7:0/1037338569 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269ffff640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f26a00a41b0 0x7f26a0140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51086/0 (socket says 172.21.15.7:51086) 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796462079 0 0) 0x7f26a00a2f50 con 0x7f26a00a41b0 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): my global_id is 96077 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2688003520 con 0x7f26a00a41b0 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1407067378 0 0) 0x7f26a00a1670 con 0x7f26a00a49a0 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): my global_id is 124638 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a00a2f50 con 0x7f26a00a49a0 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2087167458 0 0) 0x7f26a00a2c70 con 0x7f26a0137940 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient(hunting): my global_id is 6490 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a00a1670 con 0x7f26a0137940 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2598175177 0 0) 0x7f26a00a2f50 con 0x7f26a00a49a0 2024-05-08T15:55:40.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 >> v1:172.21.15.7:6790/0 conn(0x7f26a00a41b0 legacy=0x7f26a0140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 >> v1:172.21.15.81:6789/0 conn(0x7f26a0137940 legacy=0x7f26a0137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 monclient: found mon.noname-d 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a00a39f0 con 0x7f26a00a49a0 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 5 monclient: authenticate success, global_id 124638 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2694001790 con 0x7f26a00a49a0 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: dump: 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.306 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2694001a70 con 0x7f26a00a49a0 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 1 -- 172.21.15.7:0/1037338569 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2694004be0 con 0x7f26a00a49a0 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f26a5a8f640 4 set_mon_vals no callback set 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.297+0000 7f269e7fc640 10 monclient: dump: 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.307 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 10 monclient: shutdown 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 1 -- 172.21.15.7:0/1037338569 >> v1:172.21.15.7:6792/0 conn(0x7f26a00a49a0 legacy=0x7f26a013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 1 -- 172.21.15.7:0/1037338569 shutdown_connections 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 1 -- 172.21.15.7:0/1037338569 >> 172.21.15.7:0/1037338569 conn(0x7f26a00a9fa0 msgr2=0x7f26a00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 1 -- 172.21.15.7:0/1037338569 shutdown_connections 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 1 -- 172.21.15.7:0/1037338569 wait complete. 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.298+0000 7f26a5a8f640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 10 monclient: monmap: 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.309 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 1 Processor -- start 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 1 -- start start 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 10 monclient: init 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 10 monclient: _add_conns ranks=[4,2,1,3,5,0,8,7,6] 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 10 monclient(hunting): picked mon.noname-c con 0x7f26a00a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:40.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 10 monclient(hunting): picked mon.noname-b con 0x7f26a00a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 10 monclient(hunting): picked mon.noname-f con 0x7f26a0137940 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.299+0000 7f26a5a8f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a00a1f30 con 0x7f26a0137940 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a00a0f60 con 0x7f26a00a49a0 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a0133970 con 0x7f26a00a41b0 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 10 monclient(hunting): authenticate will time out at 1348.414647s 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f269f7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f26a00a41b0 0x7f26a0131f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51634/0 (socket says 172.21.15.7:51634) 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f269effd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f26a00a49a0 0x7f26a0132680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51096/0 (socket says 172.21.15.7:51096) 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f269f7fe640 1 -- 172.21.15.7:0/2407217020 learned_addr learned my addr 172.21.15.7:0/2407217020 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1687553125 0 0) 0x7f26a00a0f60 con 0x7f26a00a49a0 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): my global_id is 96086 2024-05-08T15:55:40.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f268c003520 con 0x7f26a00a49a0 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892008875 0 0) 0x7f26a0133970 con 0x7f26a00a41b0 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): my global_id is 124474 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a00a0f60 con 0x7f26a00a41b0 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340281907 0 0) 0x7f26a00a1f30 con 0x7f26a0137940 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): my global_id is 6499 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a0133970 con 0x7f26a0137940 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4148134788 0 0) 0x7f268c003520 con 0x7f26a00a49a0 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26a00a1f30 con 0x7f26a00a49a0 2024-05-08T15:55:40.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2694001690 con 0x7f26a00a49a0 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): dump: 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.313 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 842131752 0 0) 0x7f26a00a0f60 con 0x7f26a00a41b0 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f268c003520 con 0x7f26a00a41b0 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2690001970 con 0x7f26a00a41b0 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient(hunting): dump: 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.314 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 185767725 0 0) 0x7f26a0133970 con 0x7f26a0137940 2024-05-08T15:55:40.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26a00a0f60 con 0x7f26a0137940 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1042804720 0 0) 0x7f26a00a1f30 con 0x7f26a00a49a0 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.7:6791/0 conn(0x7f26a00a41b0 legacy=0x7f26a0131f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.81:6789/0 conn(0x7f26a0137940 legacy=0x7f26a01d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 monclient: found mon.b 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a0132d90 con 0x7f26a00a49a0 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f2687fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 5 monclient: authenticate success, global_id 96086 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 10 monclient: _renew_subs 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:40.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f26a01359f0 con 0x7f26a00a49a0 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 10 monclient: _renew_subs 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.300+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f26a01340c0 con 0x7f26a00a49a0 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f2694001a70 con 0x7f26a00a49a0 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2694005230 con 0x7f26a00a49a0 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f269dffb640 4 set_mon_vals no callback set 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 10 monclient: dump: 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.317 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f26940105b0 con 0x7f26a00a49a0 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f26a5a8f640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f26800065c0 con 0x7f2680002860 2024-05-08T15:55:40.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.301+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f2694046cf0 con 0x7f26a00a49a0 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f269f7fe640 10 monclient: get_auth_request con 0x7f268c03a360 auth_method 0 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178112082 0 0) 0x7f26800065c0 con 0x7f2680002860 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 10 monclient(hunting): my global_id is 6710 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26a00a1f30 con 0x7f2680002860 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 159689727 0 0) 0x7f26a00a1f30 con 0x7f2680002860 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f26800065c0 con 0x7f2680002860 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2698004ea0 con 0x7f2680002860 2024-05-08T15:55:40.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:40.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 10 monclient: dump: 2024-05-08T15:55:40.320 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.320 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.320 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.320 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 845771257 0 0) 0x7f26800065c0 con 0x7f2680002860 2024-05-08T15:55:40.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.302+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f268000a150 con 0x7f2680002860 2024-05-08T15:55:40.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.303+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f2698005060 con 0x7f2680002860 2024-05-08T15:55:40.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.303+0000 7f2687fff640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.303+0000 7f2687fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:40.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.303+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.81:6791/0 conn(0x7f2680002860 legacy=0x7f2680004d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.303+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.81:6791/0 conn(0x7f2680002860 legacy=0x7f2680004d00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.309+0000 7f26a5a8f640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.309+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f2680009e40 con 0x7f2680009100 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1601206728 0 0) 0x7f2680009e40 con 0x7f2680009100 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 10 monclient(hunting): my global_id is 6719 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26800065c0 con 0x7f2680009100 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 3691321183 0 0) 0x7f26800065c0 con 0x7f2680009100 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f2680009e40 con 0x7f2680009100 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f2698005840 con 0x7f2680009100 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 10 monclient: got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 10 monclient: dump: 2024-05-08T15:55:40.324 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.325 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1366835045 0 0) 0x7f2680009e40 con 0x7f2680009100 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 --> v1:172.21.15.81:6791/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f268000c3d0 con 0x7f2680009100 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.310+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 <== mon.5 v1:172.21.15.81:6791/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 2898790273) 0x7f2698007be0 con 0x7f2680009100 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f2687fff640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f2687fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.81:6791/0 conn(0x7f2680009100 legacy=0x7f2680009590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f2687fff640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.81:6791/0 conn(0x7f2680009100 legacy=0x7f2680009590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:40.326 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 532616, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.327 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.328 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.329 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:40.330 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.331 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:40.333 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:40.335 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.336 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.337 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.339 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:40.340 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.342 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.343 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:40.344 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.81:6812/847627118 conn(0x7f268c03a360 legacy=0x7f268c03c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 10 monclient: shutdown 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 >> v1:172.21.15.7:6790/0 conn(0x7f26a00a49a0 legacy=0x7f26a0132680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f269ffff640 1 -- 172.21.15.7:0/2407217020 reap_dead start 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f269ffff640 1 -- 172.21.15.7:0/2407217020 reap_dead start 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 shutdown_connections 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 >> 172.21.15.7:0/2407217020 conn(0x7f26a00a9fa0 msgr2=0x7f26a0130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 shutdown_connections 2024-05-08T15:55:40.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.311+0000 7f26a5a8f640 1 -- 172.21.15.7:0/2407217020 wait complete. 2024-05-08T15:55:40.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:40.347 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 9 for mon i 2024-05-08T15:55:40.347 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.i mon_status 2024-05-08T15:55:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.454+0000 7f8e6c6f8640 10 monclient: get_monmap_and_config 2024-05-08T15:55:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.454+0000 7f8e6c6f8640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.454+0000 7f8e6c6f8640 10 monclient: monmap: 2024-05-08T15:55:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.450 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:40.455569+0000 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:40.455569+0000 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.451 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 1 Processor -- start 2024-05-08T15:55:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 1 -- start start 2024-05-08T15:55:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient: init 2024-05-08T15:55:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient: _add_conns ranks=[3,7,0,6,8,4,2,1,5] 2024-05-08T15:55:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient(hunting): picked mon.noname-g con 0x7f8e64137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient(hunting): picked mon.noname-i con 0x7f8e640a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient(hunting): picked mon.noname-a con 0x7f8e640a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640a2c70 con 0x7f8e640a41b0 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640a2f50 con 0x7f8e64137940 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640a1670 con 0x7f8e640a49a0 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.455+0000 7f8e6c6f8640 10 monclient(hunting): authenticate will time out at 1348.570537s 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e6ac95640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e640a41b0 0x7f8e64140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34012/0 (socket says 172.21.15.7:34012) 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e6ac95640 1 -- 172.21.15.7:0/2051116761 learned_addr learned my addr 172.21.15.7:0/2051116761 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 401852347 0 0) 0x7f8e640a2c70 con 0x7f8e640a41b0 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): my global_id is 95859 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e50003520 con 0x7f8e640a41b0 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1105683981 0 0) 0x7f8e640a2f50 con 0x7f8e64137940 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient(hunting): my global_id is 76953 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e640a2c70 con 0x7f8e64137940 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3587655851 0 0) 0x7f8e50003520 con 0x7f8e640a41b0 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 >> v1:172.21.15.81:6792/0 conn(0x7f8e640a49a0 legacy=0x7f8e6413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 >> v1:172.21.15.81:6790/0 conn(0x7f8e64137940 legacy=0x7f8e64137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 monclient: found mon.noname-a 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e640a39f0 con 0x7f8e640a41b0 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e5c001b30 con 0x7f8e640a41b0 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: dump: 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e6c6f8640 5 monclient: authenticate success, global_id 95859 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e6c6f8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8e5c001e50 con 0x7f8e640a41b0 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 1 -- 172.21.15.7:0/2051116761 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e5c0020c0 con 0x7f8e640a41b0 2024-05-08T15:55:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e69492640 10 monclient: dump: 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.465 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e6c6f8640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.456+0000 7f8e6c6f8640 4 set_mon_vals no callback set 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 10 monclient: shutdown 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/2051116761 >> v1:172.21.15.7:6789/0 conn(0x7f8e640a41b0 legacy=0x7f8e64140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/2051116761 shutdown_connections 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/2051116761 >> 172.21.15.7:0/2051116761 conn(0x7f8e640a9fa0 msgr2=0x7f8e640aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/2051116761 shutdown_connections 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/2051116761 wait complete. 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.457+0000 7f8e6c6f8640 10 monclient: monmap: 2024-05-08T15:55:40.466 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.467 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 1 Processor -- start 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 1 -- start start 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient: init 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient: _add_conns ranks=[0,8,6,4,7,3,2,5,1] 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient(hunting): picked mon.noname-a con 0x7f8e640a41b0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient(hunting): picked mon.noname-e con 0x7f8e640a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient(hunting): picked mon.noname-d con 0x7f8e64137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e64137760 con 0x7f8e640a41b0 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640a1f30 con 0x7f8e64137940 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e640a0f60 con 0x7f8e640a49a0 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6c6f8640 10 monclient(hunting): authenticate will time out at 1348.573425s 2024-05-08T15:55:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6a494640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f8e640a41b0 0x7f8e64137050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34028/0 (socket says 172.21.15.7:34028) 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.458+0000 7f8e6a494640 1 -- 172.21.15.7:0/1023412228 learned_addr learned my addr 172.21.15.7:0/1023412228 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 876882613 0 0) 0x7f8e640a2c70 con 0x7f8e640a41b0 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): my global_id is 95868 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e48003520 con 0x7f8e640a41b0 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2388734881 0 0) 0x7f8e640a0f60 con 0x7f8e640a49a0 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): my global_id is 144179 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e640a2c70 con 0x7f8e640a49a0 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3796026539 0 0) 0x7f8e640a1f30 con 0x7f8e64137940 2024-05-08T15:55:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): my global_id is 124647 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e640a0f60 con 0x7f8e64137940 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3351894803 0 0) 0x7f8e48003520 con 0x7f8e640a41b0 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e640a1f30 con 0x7f8e640a41b0 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e600019f0 con 0x7f8e640a41b0 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): dump: 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.470 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1166207517 0 0) 0x7f8e640a2c70 con 0x7f8e640a49a0 2024-05-08T15:55:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e48003520 con 0x7f8e640a49a0 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1450961859 0 0) 0x7f8e640a0f60 con 0x7f8e64137940 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e640a2c70 con 0x7f8e64137940 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e54001990 con 0x7f8e640a49a0 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): dump: 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e5c001af0 con 0x7f8e64137940 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient(hunting): dump: 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 978087953 0 0) 0x7f8e640a1f30 con 0x7f8e640a41b0 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.7:6793/0 conn(0x7f8e640a49a0 legacy=0x7f8e641d1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.7:6792/0 conn(0x7f8e64137940 legacy=0x7f8e641d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 monclient: found mon.a 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e64133f10 con 0x7f8e640a41b0 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e6c6f8640 5 monclient: authenticate success, global_id 95868 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e6c6f8640 10 monclient: _renew_subs 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e6c6f8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8e641345b0 con 0x7f8e640a41b0 2024-05-08T15:55:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e6c6f8640 10 monclient: _renew_subs 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e6c6f8640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8e60001d30 con 0x7f8e640a41b0 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f8e641c90b0 con 0x7f8e640a41b0 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e60003c00 con 0x7f8e640a41b0 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e68c91640 4 set_mon_vals no callback set 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.459+0000 7f8e5affd640 10 monclient: dump: 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.460+0000 7f8e6c6f8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.460+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8e640753d0 con 0x7f8e6409b580 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.460+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f8e60003e20 con 0x7f8e640a41b0 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.460+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f8e60046d40 con 0x7f8e640a41b0 2024-05-08T15:55:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e6ac95640 10 monclient: get_auth_request con 0x7f8e4803a540 auth_method 0 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175163619 0 0) 0x7f8e640753d0 con 0x7f8e6409b580 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient(hunting): my global_id is 115261 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e640a1f30 con 0x7f8e6409b580 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 1582354385 0 0) 0x7f8e640a1f30 con 0x7f8e6409b580 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e640753d0 con 0x7f8e6409b580 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e54003930 con 0x7f8e6409b580 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient: dump: 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 3595256894 0 0) 0x7f8e640753d0 con 0x7f8e6409b580 2024-05-08T15:55:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e64133850 con 0x7f8e6409b580 2024-05-08T15:55:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (unknown 0 0 4045327437) 0x7f8e54005bd0 con 0x7f8e6409b580 2024-05-08T15:55:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient: handle_command_reply 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.461+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.81:6792/0 conn(0x7f8e6409b580 legacy=0x7f8e6409b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.462+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.81:6792/0 conn(0x7f8e6409b580 legacy=0x7f8e6409b960 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.468+0000 7f8e6c6f8640 10 start_mon_command cmd=[{"prefix": "mon_status"}] 2024-05-08T15:55:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.468+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 1) -- 0x7f8e641347e0 con 0x7f8e6409c070 2024-05-08T15:55:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 782176781 0 0) 0x7f8e641347e0 con 0x7f8e6409c070 2024-05-08T15:55:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient(hunting): my global_id is 115270 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e640753d0 con 0x7f8e6409c070 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 390+0+0 (unknown 367882665 0 0) 0x7f8e640753d0 con 0x7f8e6409c070 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8e641347e0 con 0x7f8e6409c070 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8e540054d0 con 0x7f8e6409c070 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient: got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient: dump: 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 206+0+0 (unknown 1176469106 0 0) 0x7f8e641347e0 con 0x7f8e6409c070 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 --> v1:172.21.15.81:6792/0 -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8e64141d00 con 0x7f8e6409c070 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 <== mon.7 v1:172.21.15.81:6792/0 5 ==== command_reply(tid 2: 0 ) ==== 8+0+7693 (unknown 0 0 2635829292) 0x7f8e54005700 con 0x7f8e6409c070 2024-05-08T15:55:40.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient: handle_command_reply 2 [{"prefix": "mon_status"}] 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.81:6792/0 conn(0x7f8e6409c070 legacy=0x7f8e641da5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.469+0000 7f8e5affd640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.81:6792/0 conn(0x7f8e6409c070 legacy=0x7f8e641da5a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "peon", 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "election_epoch": 60, 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 275333, 2024-05-08T15:55:40.484 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum": [ 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 0, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 1, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 2, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 3, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 4, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 5, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 6, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 7, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: 8 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_age": 1, 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "required_con": "2450521708007866372", 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "required_mon": [ 2024-05-08T15:55:40.485 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_con": "4540701547738038271", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "quorum_mon": [ 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.486 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "outside_quorum": [], 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "extra_probe_peers": [], 2024-05-08T15:55:40.487 INFO:teuthology.orchestra.run.smithi007.stdout: "sync_provider": [], 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "monmap": { 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "epoch": 1, 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "fsid": "402a637b-159c-4d0b-9f9c-7ff72b154b33", 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "modified": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "created": "2024-05-08T15:46:23.145395Z", 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release": 19, 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "min_mon_release_name": "squid", 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "election_strategy": 3, 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "disallowed_leaders: ": "", 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false, 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "tiebreaker_mon": "", 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "removed_ranks: ": "", 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "features": { 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "persistent": [ 2024-05-08T15:55:40.488 INFO:teuthology.orchestra.run.smithi007.stdout: "kraken", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "luminous", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "mimic", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap-prune", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "nautilus", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "octopus", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "pacific", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "elector-pinging", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "quincy", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "reef", 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "squid" 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "optional": [] 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: "mons": [ 2024-05-08T15:55:40.489 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 0, 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "a", 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789", 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6789/0", 2024-05-08T15:55:40.490 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 1, 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "f", 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789", 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.491 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6789/0", 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 2, 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "b", 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790", 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.492 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6790/0", 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 3, 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "g", 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.493 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790", 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6790/0", 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 4, 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "c", 2024-05-08T15:55:40.494 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791", 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6791/0", 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.495 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 5, 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "h", 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791", 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6791/0", 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.496 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 6, 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "d", 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792", 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.497 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6792/0", 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 7, 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "i", 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792", 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.498 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.81:6792/0", 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "rank": 8, 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "name": "e", 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addrs": { 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "addrvec": [ 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.499 INFO:teuthology.orchestra.run.smithi007.stdout: "type": "v1", 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793", 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "nonce": 0 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "public_addr": "172.21.15.7:6793/0", 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "priority": 0, 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "weight": 0, 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "crush_location": "{}" 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "feature_map": { 2024-05-08T15:55:40.500 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": [ 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": [ 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 2 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: "client": [ 2024-05-08T15:55:40.501 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "release": "luminous", 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "num": 1 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout: "stretch_mode": false 2024-05-08T15:55:40.502 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.81:6812/847627118 conn(0x7f8e4803a540 legacy=0x7f8e4803ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 10 monclient: shutdown 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 >> v1:172.21.15.7:6789/0 conn(0x7f8e640a41b0 legacy=0x7f8e64137050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6ac95640 1 -- 172.21.15.7:0/1023412228 reap_dead start 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6ac95640 1 -- 172.21.15.7:0/1023412228 reap_dead start 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 shutdown_connections 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 >> 172.21.15.7:0/1023412228 conn(0x7f8e640a9fa0 msgr2=0x7f8e64130a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 shutdown_connections 2024-05-08T15:55:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.470+0000 7f8e6c6f8640 1 -- 172.21.15.7:0/1023412228 wait complete. 2024-05-08T15:55:40.504 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 9 2024-05-08T15:55:40.504 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-05-08T15:55:40.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-05-08T15:55:40.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.612+0000 7f05bfe7b640 10 monclient: get_monmap_and_config 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.612+0000 7f05bfe7b640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.612+0000 7f05bfe7b640 10 monclient: monmap: 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:40.613064+0000 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:40.613064+0000 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.608 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.609 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.609 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.612+0000 7f05bfe7b640 1 Processor -- start 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.612+0000 7f05bfe7b640 1 -- start start 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.612+0000 7f05bfe7b640 10 monclient: init 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 10 monclient: _add_conns ranks=[0,8,7,1,6,2,4,5,3] 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 10 monclient(hunting): picked mon.noname-a con 0x7f05b80a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 10 monclient(hunting): picked mon.noname-e con 0x7f05b80a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:40.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 10 monclient(hunting): picked mon.noname-i con 0x7f05b813e310 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b80a1660 con 0x7f05b80a53a0 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b80a28f0 con 0x7f05b813e310 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b80a2bd0 con 0x7f05b80a5780 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bdc17640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f05b80a53a0 0x7f05b80a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34032/0 (socket says 172.21.15.7:34032) 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bfe7b640 10 monclient(hunting): authenticate will time out at 1348.728250s 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bdc17640 1 -- 172.21.15.7:0/2954150938 learned_addr learned my addr 172.21.15.7:0/2954150938 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bd416640 1 --1- 172.21.15.7:0/2954150938 >> v1:172.21.15.7:6793/0 conn(0x7f05b80a5780 0x7f05b813cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55896/0 (socket says 172.21.15.7:55896) 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4016958639 0 0) 0x7f05b80a1660 con 0x7f05b80a53a0 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): my global_id is 95877 2024-05-08T15:55:40.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05a4003520 con 0x7f05b80a53a0 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3511929073 0 0) 0x7f05b80a2bd0 con 0x7f05b80a5780 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 10 monclient(hunting): my global_id is 144188 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.613+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b80a1660 con 0x7f05b80a5780 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 444043254 0 0) 0x7f05a4003520 con 0x7f05b80a53a0 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 >> v1:172.21.15.7:6793/0 conn(0x7f05b80a5780 legacy=0x7f05b813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 >> v1:172.21.15.81:6792/0 conn(0x7f05b813e310 legacy=0x7f05b8140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 monclient: found mon.noname-a 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b80a6730 con 0x7f05b80a53a0 2024-05-08T15:55:40.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f05b40016e0 con 0x7f05b80a53a0 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 5 monclient: authenticate success, global_id 95877 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: dump: 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.620 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f05b40019c0 con 0x7f05b80a53a0 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 1 -- 172.21.15.7:0/2954150938 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f05b4004be0 con 0x7f05b80a53a0 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 4 set_mon_vals no callback set 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bcc15640 10 monclient: dump: 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 10 monclient: shutdown 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 1 -- 172.21.15.7:0/2954150938 >> v1:172.21.15.7:6789/0 conn(0x7f05b80a53a0 legacy=0x7f05b80a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 1 -- 172.21.15.7:0/2954150938 shutdown_connections 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.614+0000 7f05bfe7b640 1 -- 172.21.15.7:0/2954150938 >> 172.21.15.7:0/2954150938 conn(0x7f05b80a9fa0 msgr2=0x7f05b80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.615+0000 7f05bfe7b640 1 -- 172.21.15.7:0/2954150938 shutdown_connections 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.615+0000 7f05bfe7b640 1 -- 172.21.15.7:0/2954150938 wait complete. 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.615+0000 7f05bfe7b640 10 monclient: build_initial_monmap 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.615+0000 7f05bfe7b640 10 monclient: monmap: 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.615+0000 7f05bfe7b640 1 Processor -- start 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.615+0000 7f05bfe7b640 1 -- start start 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.615+0000 7f05bfe7b640 10 monclient: init 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 10 monclient: _add_conns ranks=[7,8,6,1,3,4,0,5,2] 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 10 monclient(hunting): picked mon.noname-i con 0x7f05b80a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 10 monclient(hunting): picked mon.noname-e con 0x7f05b80a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 10 monclient(hunting): picked mon.noname-d con 0x7f05b813e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b8137e50 con 0x7f05b813e310 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b80a1e00 con 0x7f05b80a53a0 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05b80a43c0 con 0x7f05b80a5780 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bfe7b640 10 monclient(hunting): authenticate will time out at 1348.731135s 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bd416640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f05b80a5780 0x7f05b81345a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55902/0 (socket says 172.21.15.7:55902) 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05bd416640 1 -- 172.21.15.7:0/66598566 learned_addr learned my addr 172.21.15.7:0/66598566 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2090178667 0 0) 0x7f05b80a43c0 con 0x7f05b80a5780 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): my global_id is 144197 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f059c003520 con 0x7f05b80a5780 2024-05-08T15:55:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3915706762 0 0) 0x7f05b8137e50 con 0x7f05b813e310 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 10 monclient(hunting): my global_id is 124656 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.616+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b80a43c0 con 0x7f05b813e310 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2577088305 0 0) 0x7f05b80a1e00 con 0x7f05b80a53a0 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): my global_id is 115288 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05b8137e50 con 0x7f05b80a53a0 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3712582248 0 0) 0x7f059c003520 con 0x7f05b80a5780 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f05b80a1e00 con 0x7f05b80a5780 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f05a80014d0 con 0x7f05b80a5780 2024-05-08T15:55:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2026361600 0 0) 0x7f05b80a43c0 con 0x7f05b813e310 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f059c003520 con 0x7f05b813e310 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f05b0001c30 con 0x7f05b813e310 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05be418640 1 -- 172.21.15.7:0/66598566 >> v1:172.21.15.7:6792/0 conn(0x7f05b813e310 legacy=0x7f05b81d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05be418640 1 -- 172.21.15.7:0/66598566 >> v1:172.21.15.7:6792/0 conn(0x7f05b813e310 legacy=0x7f05b81d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T15:55:40.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05be418640 1 --1- 172.21.15.7:0/66598566 >> v1:172.21.15.7:6792/0 conn(0x7f05b813e310 0x7f05b81d4d60 :-1 s=OPENED pgs=413 cs=1 l=1).handle_message read tag failed 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05be418640 1 --1- 172.21.15.7:0/66598566 >> v1:172.21.15.7:6792/0 conn(0x7f05b813e310 0x7f05b81d4d60 :-1 s=OPENED pgs=413 cs=1 l=1).fault on lossy channel, failing 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.7:6792/0 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3278009571 0 0) 0x7f05b8137e50 con 0x7f05b80a53a0 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f05b80a43c0 con 0x7f05b80a53a0 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f05b40016a0 con 0x7f05b80a53a0 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:40.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:40.632 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.632 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.632 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 676508531 0 0) 0x7f05b80a1e00 con 0x7f05b80a5780 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 >> v1:172.21.15.81:6792/0 conn(0x7f05b80a53a0 legacy=0x7f05b8133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 >> v1:172.21.15.7:6792/0 conn(0x7f05b813e310 legacy=0x7f05b81d4d60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 monclient: found mon.e 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b8135b60 con 0x7f05b80a5780 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05bfe7b640 5 monclient: authenticate success, global_id 144197 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05bfe7b640 10 monclient: _renew_subs 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05bfe7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:40.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f05b8137b70 con 0x7f05b80a5780 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05bfe7b640 10 monclient: _renew_subs 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05bfe7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f05b81385c0 con 0x7f05b80a5780 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f05a80017d0 con 0x7f05b80a5780 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f05a8004dd0 con 0x7f05b80a5780 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05affff640 4 set_mon_vals no callback set 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.617+0000 7f05ae7fc640 10 monclient: dump: 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:40.635 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.618+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f05a8010150 con 0x7f05b80a5780 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.618+0000 7f05bfe7b640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.618+0000 7f05bfe7b640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.618+0000 7f05bfe7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:40.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.618+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a0005080 con 0x7f05b80a5780 2024-05-08T15:55:40.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.618+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f05a8046c70 con 0x7f05b80a5780 2024-05-08T15:55:40.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.618+0000 7f05bdc17640 10 monclient: get_auth_request con 0x7f059c03a3f0 auth_method 0 2024-05-08T15:55:40.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.623+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f05a8015e00 con 0x7f05b80a5780 2024-05-08T15:55:40.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.623+0000 7f05ae7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:40.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.623+0000 7f05ae7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.886+0000 7f05bfe7b640 10 start_mon_command cmd=[{"prefix": "mon scrub"}] 2024-05-08T15:55:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.886+0000 7f05bfe7b640 10 monclient: _send_command 2 [{"prefix": "mon scrub"}] 2024-05-08T15:55:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.886+0000 7f05bfe7b640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:40.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.886+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f05a0005b80 con 0x7f05b80a5780 2024-05-08T15:55:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.888+0000 7f05ae7fc640 1 -- 172.21.15.7:0/66598566 <== mon.8 v1:172.21.15.7:6793/0 10 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (unknown 246732539 0 0) 0x7f05a8015700 con 0x7f05b80a5780 2024-05-08T15:55:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.888+0000 7f05ae7fc640 10 monclient: handle_mon_command_ack 2 [{"prefix": "mon scrub"}] 2024-05-08T15:55:40.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.888+0000 7f05ae7fc640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.890+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 >> v1:172.21.15.81:6812/847627118 conn(0x7f059c03a3f0 legacy=0x7f059c03c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.890+0000 7f05bfe7b640 10 monclient: shutdown 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.890+0000 7f05bfe7b640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.890+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 >> v1:172.21.15.7:6793/0 conn(0x7f05b80a5780 legacy=0x7f05b81345a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.890+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 shutdown_connections 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.890+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 >> 172.21.15.7:0/66598566 conn(0x7f05b80a9fa0 msgr2=0x7f05b80a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.891+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 shutdown_connections 2024-05-08T15:55:40.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:40.891+0000 7f05bfe7b640 1 -- 172.21.15.7:0/66598566 wait complete. 2024-05-08T15:55:40.904 INFO:tasks.mon_thrash.mon_thrasher:waiting for 1.0 secs before continuing thrashing 2024-05-08T15:55:40.961 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.961 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.961 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {auth=22,config=2,health=12,kv=9,logm=55} crc {auth=480336418,config=1618278320,health=1378089662,kv=1462314268,logm=816337300}) 2024-05-08T15:55:40.961 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.961 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.961 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {auth=21,config=2,health=11,kv=10,logm=56} crc {auth=3918713632,config=1618278320,health=3769616292,kv=3560234957,logm=2621238773}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=3180305651,config=1618278320,health=1378089662,kv=3560234957,logm=589293949}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {auth=20,config=2,health=12,kv=10,logm=56} crc {auth=2163892860,config=1618278320,health=1378089662,kv=3560234957,logm=1842401464}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {auth=22,config=2,health=12,kv=10,logm=54} crc {auth=480336418,config=1618278320,health=1378089662,kv=3560234957,logm=3588207163}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=955087986,config=1618278320,health=1378089662,kv=3560234957,logm=120191431}) 2024-05-08T15:55:40.962 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {auth=21,config=2,health=11,kv=9,logm=57} crc {auth=2771182359,config=1618278320,health=1435182105,kv=1462314268,logm=2336090589}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {auth=21,config=2,health=12,kv=10,logm=55} crc {auth=1741394928,config=1618278320,health=2985365779,kv=3560234957,logm=450114689}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.888+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {auth=22,config=2,health=11,kv=10,logm=55} crc {auth=480336418,config=1618278320,health=3550891638,kv=3560234957,logm=2976503065}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3982828008}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2845757613}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.963 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1410040716}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2833840992}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3472762057}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=78618169}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2207529721}) 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.964 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1552165947}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.890+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3416854554}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.891+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.891+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.891+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=285449749}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.891+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.891+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1119686146}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3105866087}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.965 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2204617055}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3727191776}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3692365140}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1952393546}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1609154220}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.892+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3124116223}) 2024-05-08T15:55:40.966 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.893+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.893+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.893+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2280031536}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.893+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.893+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1699021373}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2397908423}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4197932244}) 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.967 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3720244592}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=910637332}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=486369580}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=896193813}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.894+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1583074622}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=843038843}) 2024-05-08T15:55:40.968 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1092013353}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2173066696}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3817263046}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=451418501}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.969 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3792164638}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1940142829}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3741818005}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.895+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4226368004}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3815309525}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3336085534}) 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.970 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4128827142}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=810114784}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4147890798}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=343444638}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3276354720}) 2024-05-08T15:55:40.971 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1195439788}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.897+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2186073195}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1128190774}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1828359225}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2760399107}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.972 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2375162941}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.898+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3513887960}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3828463557}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3329825411}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=4183782545}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.899+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2037287483}) 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.973 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3693337913}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=423123730}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=804580810}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3066620412}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4154224141}) 2024-05-08T15:55:40.974 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=4267916118}) 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3089425020}) 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3675622002}) 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.900+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3127923109}) 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.975 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=989388310}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=217578139}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.901+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3460084051}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3778435892}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.976 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3942530819}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2125138547}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1818091764}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2424675482}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.902+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2608310954}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4136652495}) 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.977 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=882318007}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3659757576}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1869049151}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2291121374}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.978 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846751711}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1278780285}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3435346684}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.903+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3746938749}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1614186575}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3805924834}) 2024-05-08T15:55:40.979 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=796622942}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3285068728}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=125474406}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1475165221}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.980 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3314414891}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3860285074}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.905+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3806378312}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=232748087}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=4136450550}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=879450200}) 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.981 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3094259580}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2139891796}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.906+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.907+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=24403754}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.907+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.907+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.907+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1997741590}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.907+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.907+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3721985772}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.907+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1630381831}) 2024-05-08T15:55:40.982 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=674222888}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=651706887}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=221026867}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2470341633}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.983 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=885142389}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=196881251}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=628190090}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3907335467}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.908+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1557616092}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.984 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3497404072}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1228245917}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3612367640}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.909+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4197620803}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1362216609}) 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.985 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1297996519}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2151151259}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3069678300}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.910+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1400754238}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1416949397}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.986 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1912451258}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1699896490}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3094842789}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=887698795}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2850184677}) 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.987 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=37656084}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2307245755}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.911+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3320490505}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.912+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.912+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.912+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1873840359}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.912+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.912+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3194699302}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1391263959}) 2024-05-08T15:55:40.988 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3264208047}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3939479769}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3311442871}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4252511616}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3029807085}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.913+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3199318335}) 2024-05-08T15:55:40.989 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=315024303}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=655878859}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=329938268}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3429127173}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.990 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1235372187}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1483440549}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3944759154}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3580602486}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.914+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1654620889}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=4086296190}) 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.991 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=922386924}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=22451956}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1033200791}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4004778985}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=277870470}) 2024-05-08T15:55:40.992 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3871571328}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2825603713}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.916+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1246104747}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=959820999}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=189316923}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2604481037}) 2024-05-08T15:55:40.993 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3344381442}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=83536361}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3952564261}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1500687758}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.917+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.918+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=767323111}) 2024-05-08T15:55:40.994 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.918+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4024955593}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1772953313}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3674370987}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=4288650339}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3118339567}) 2024-05-08T15:55:40.995 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1225743182}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1654623080}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3112275404}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=59133920}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.919+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2065091785}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.996 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1308419945}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3214722456}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3255299331}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1771296614}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=251819141}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.997 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=558421704}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.920+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.921+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=4232215119}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.921+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.921+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3958310816}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.921+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=4202386670}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=696234009}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1182847023}) 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.998 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1920906070}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2800817939}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2491279125}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2846542302}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2643337785}) 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:40.999 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1304604821}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.922+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1659087270}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2538144003}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=446273247}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=365506858}) 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.000 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3331111152}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1019426723}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.923+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=651379408}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.924+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.924+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.924+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=1647292187}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.924+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.924+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=3451544556}) 2024-05-08T15:55:41.001 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.924+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=500671021}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2058946960}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3265328736}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2307845805}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3857174738}) 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.002 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=4036632430}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1274656148}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3474027522}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1410720947}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.925+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3466357451}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1422334398}) 2024-05-08T15:55:41.003 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3171253415}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3904319738}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=1243904857}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=1674629304}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.004 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=2563494715}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2995411453}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2943266663}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.926+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=3403351153}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=3469438707}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3752133102}) 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.005 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3057915103}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=2388425574}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.927+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=2237776175}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3981924314}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=3146147376}) 2024-05-08T15:55:41.006 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=986193956}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.928+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=619516237}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=1148864143}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2978159217}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=341335512}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.007 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3643755424}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=54514790}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=1414984315}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2799575770}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1279837547}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.929+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=2232018784}) 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.008 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=2221196827}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=3557572341}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=3053376200}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=3090800061}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.009 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3925503983}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.930+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3610470729}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.931+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.931+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.931+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=2031137017}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.931+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.931+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2950278042}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.931+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=544991056}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=973071244}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.010 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=1105309835}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=2131768970}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4005837413}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3076054562}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=3407924616}) 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.011 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=158383984}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=1846664104}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.932+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=1423042983}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=100} crc {logm=494055741}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=100} crc {logm=2861102018}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=100} crc {logm=1321935684}) 2024-05-08T15:55:41.012 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=100} crc {logm=4226473348}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=100} crc {logm=3083342537}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=100} crc {logm=69120061}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.933+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.934+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.934+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=100} crc {logm=426681156}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.934+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.934+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=100} crc {logm=2086846644}) 2024-05-08T15:55:41.013 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.934+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=100} crc {logm=183142047}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {logm=59,mdsmap=3,mgr=6,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7,monmap=2,osd_metadata=6,osdmap=5} crc {logm=332378741,mdsmap=2840952586,mgr=1238733155,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=4221702291,mon_config_key=2368181095,monmap=3140514009,osd_metadata=1026190808,osdmap=792411391}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {logm=61,mdsmap=3,mgr=8,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=5} crc {logm=1274849947,mdsmap=3764978221,mgr=4070169375,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=4221702292,mon_config_key=619004784,monmap=256126145,osd_metadata=307955154}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {logm=59,mdsmap=2,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=2} crc {logm=1076919365,mdsmap=3140514009,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=1310457925,mon_config_key=619004784,monmap=256126145,osd_metadata=2470004706,osd_pg_creating=2683498337,osdmap=1247189103}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {logm=61,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=7,monmap=3,osd_metadata=6,osdmap=1} crc {logm=1828808922,mdsmap=2840952586,mgr=4015922855,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=4221702291,mon_config_key=489254662,monmap=256126145,osd_metadata=1026190808,osdmap=3132431388}) 2024-05-08T15:55:41.014 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {logm=61,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6} crc {logm=852564792,mdsmap=2840952586,mgr=4015922855,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=4221702291,mon_config_key=3665297902,monmap=256126145,osd_metadata=1026190808}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {logm=59,mdsmap=2,mgr=8,mgrstat=10,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=3} crc {logm=3651623621,mdsmap=1401145748,mgr=584111586,mgrstat=4221702291,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=1209664841}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {logm=60,mdsmap=2,mgr=7,mgr_metadata=1,mgrstat=10,mon_config_key=7,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=3} crc {logm=4288542182,mdsmap=1401145748,mgr=4015922855,mgr_metadata=1815193447,mgrstat=4221702291,mon_config_key=1403364458,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=1209664841}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {logm=60,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=1} crc {logm=513674831,mdsmap=2840952586,mgr=2820595276,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2026488570,mon_config_key=789160039,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=3132431388}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.935+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {logm=58,mdsmap=3,mgr=7,mgr_command_descs=1,mgr_metadata=1,mgrstat=9,mon_config_key=8,monmap=3,osd_metadata=6,osd_pg_creating=1,osdmap=3} crc {logm=1271458811,mdsmap=2840952586,mgr=4015922855,mgr_command_descs=3353810608,mgr_metadata=1815193447,mgrstat=2727612154,mon_config_key=619004784,monmap=256126145,osd_metadata=1026190808,osd_pg_creating=2683498337,osdmap=1209664841}) 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.936+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.015 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.936+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2312425099}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=2194346797}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=2543571271}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2688187978}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=3336661195}) 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.016 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3007107826}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3367990260}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2512065021}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.937+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=2360692003}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=752568999}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=1079618193}) 2024-05-08T15:55:41.017 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=3270964657}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=1303710729}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=703783110}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=2162749852}) 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.018 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=3678688234}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=4068807264}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.938+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=4215713677}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=100} crc {osdmap=2942076957}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=100} crc {osdmap=507382522}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=100} crc {osdmap=1508563627}) 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.019 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=100} crc {osdmap=2950569482}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=100} crc {osdmap=4157223164}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=100} crc {osdmap=3073542537}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=100} crc {osdmap=2447216901}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=100} crc {osdmap=2347181342}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.940+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=100} crc {osdmap=347277246}) 2024-05-08T15:55:41.020 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.0 ScrubResult(keys {osdmap=94} crc {osdmap=2275297843}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.2 ScrubResult(keys {osdmap=91} crc {osdmap=3371606740}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.3 ScrubResult(keys {osdmap=94} crc {osdmap=3012705071}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.4 ScrubResult(keys {osdmap=93} crc {osdmap=3160179752}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.5 ScrubResult(keys {osdmap=93} crc {osdmap=920320353}) 2024-05-08T15:55:41.021 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.6 ScrubResult(keys {osdmap=92} crc {osdmap=3173534090}) 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.942+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.943+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.7 ScrubResult(keys {osdmap=95} crc {osdmap=3310718096}) 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.943+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : scrub mismatch 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.943+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.1 ScrubResult(keys {osdmap=94} crc {osdmap=2781568830}) 2024-05-08T15:55:41.022 INFO:tasks.ceph.mon.f.smithi081.stderr:2024-05-08T15:55:40.943+0000 7f54eead5640 -1 log_channel(cluster) log [ERR] : mon.8 ScrubResult(keys {osdmap=94} crc {osdmap=190177909}) 2024-05-08T15:55:41.906 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 9 2024-05-08T15:55:41.906 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T15:55:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.021+0000 7f6597579640 10 monclient: get_monmap_and_config 2024-05-08T15:55:42.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.021+0000 7f6597579640 10 monclient: build_initial_monmap 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.021+0000 7f6597579640 10 monclient: monmap: 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:42.022933+0000 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:42.022933+0000 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:42.018 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:42.019 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:42.019 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.022+0000 7f6597579640 1 Processor -- start 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.022+0000 7f6597579640 1 -- start start 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.022+0000 7f6597579640 10 monclient: init 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 10 monclient: _add_conns ranks=[2,1,7,6,0,5,4,8,3] 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 10 monclient(hunting): picked mon.noname-b con 0x7f6590137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 10 monclient(hunting): picked mon.noname-f con 0x7f65900a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 10 monclient(hunting): picked mon.noname-i con 0x7f65900a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65900a2c70 con 0x7f65900a49a0 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65900a2f50 con 0x7f6590137940 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65900a1670 con 0x7f65900a41b0 2024-05-08T15:55:42.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6597579640 10 monclient(hunting): authenticate will time out at 1350.137943s 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6595315640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f6590137940 0x7f6590137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51112/0 (socket says 172.21.15.7:51112) 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6595315640 1 -- 172.21.15.7:0/2827550223 learned_addr learned my addr 172.21.15.7:0/2827550223 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2294231735 0 0) 0x7f65900a2f50 con 0x7f6590137940 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient(hunting): my global_id is 96095 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f657c003520 con 0x7f6590137940 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 687831300 0 0) 0x7f657c003520 con 0x7f6590137940 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 >> v1:172.21.15.81:6792/0 conn(0x7f65900a41b0 legacy=0x7f6590140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 >> v1:172.21.15.81:6789/0 conn(0x7f65900a49a0 legacy=0x7f659013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 monclient: found mon.noname-b 2024-05-08T15:55:42.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65900a39f0 con 0x7f6590137940 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient: _finish_auth 0 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f658c001b30 con 0x7f6590137940 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.023+0000 7f6587fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 10 monclient: dump: 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.028 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 5 monclient: authenticate success, global_id 96095 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f658c001e90 con 0x7f6590137940 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:42.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 1 -- 172.21.15.7:0/2827550223 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f658c0020c0 con 0x7f6590137940 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 4 set_mon_vals no callback set 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6587fff640 10 monclient: dump: 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.030 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 10 monclient: shutdown 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 1 -- 172.21.15.7:0/2827550223 >> v1:172.21.15.7:6790/0 conn(0x7f6590137940 legacy=0x7f6590137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 1 -- 172.21.15.7:0/2827550223 shutdown_connections 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 1 -- 172.21.15.7:0/2827550223 >> 172.21.15.7:0/2827550223 conn(0x7f65900a9fa0 msgr2=0x7f65900aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 1 -- 172.21.15.7:0/2827550223 shutdown_connections 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.024+0000 7f6597579640 1 -- 172.21.15.7:0/2827550223 wait complete. 2024-05-08T15:55:42.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.025+0000 7f6597579640 10 monclient: build_initial_monmap 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.025+0000 7f6597579640 10 monclient: monmap: 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:42.032 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.025+0000 7f6597579640 1 Processor -- start 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.025+0000 7f6597579640 1 -- start start 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.025+0000 7f6597579640 10 monclient: init 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 10 monclient: _add_conns ranks=[1,0,4,6,3,5,7,8,2] 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 10 monclient(hunting): picked mon.noname-f con 0x7f65900a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 10 monclient(hunting): picked mon.noname-a con 0x7f65900a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 10 monclient(hunting): picked mon.noname-c con 0x7f6590137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6590135ea0 con 0x7f65900a49a0 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6590134010 con 0x7f65900a41b0 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65901cda40 con 0x7f6590137940 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6597579640 10 monclient(hunting): authenticate will time out at 1350.140711s 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6594b14640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f65900a49a0 0x7f65901cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34042/0 (socket says 172.21.15.7:34042) 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6594b14640 1 -- 172.21.15.7:0/2015355891 learned_addr learned my addr 172.21.15.7:0/2015355891 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6595315640 1 --1- 172.21.15.7:0/2015355891 >> v1:172.21.15.81:6789/0 conn(0x7f65900a41b0 0x7f65901ccc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:46322/0 (socket says 172.21.15.7:46322) 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16892337 0 0) 0x7f65901cda40 con 0x7f6590137940 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): my global_id is 124483 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6574003520 con 0x7f6590137940 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 294227371 0 0) 0x7f6590135ea0 con 0x7f65900a49a0 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): my global_id is 95886 2024-05-08T15:55:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f65901cda40 con 0x7f65900a49a0 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3852392699 0 0) 0x7f6590134010 con 0x7f65900a41b0 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): my global_id is 6517 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6590135ea0 con 0x7f65900a41b0 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4266640677 0 0) 0x7f6574003520 con 0x7f6590137940 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6590134010 con 0x7f6590137940 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 208072258 0 0) 0x7f65901cda40 con 0x7f65900a49a0 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f6574003520 con 0x7f65900a49a0 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6588001950 con 0x7f6590137940 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:55:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2228094358 0 0) 0x7f6590135ea0 con 0x7f65900a41b0 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f65901cda40 con 0x7f65900a41b0 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f65800019d0 con 0x7f65900a49a0 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f658c001af0 con 0x7f65900a41b0 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.026+0000 7f6585ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 585660742 0 0) 0x7f6590134010 con 0x7f6590137940 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 >> v1:172.21.15.81:6789/0 conn(0x7f65900a41b0 legacy=0x7f65901ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 >> v1:172.21.15.7:6789/0 conn(0x7f65900a49a0 legacy=0x7f65901cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 1 monclient: found mon.c 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65900a1f30 con 0x7f6590137940 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: _finish_auth 0 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6597579640 5 monclient: authenticate success, global_id 124483 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6597579640 10 monclient: _renew_subs 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6597579640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f6590135bc0 con 0x7f6590137940 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f6588001d20 con 0x7f6590137940 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f6588003c00 con 0x7f6590137940 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6585ffb640 10 monclient: dump: 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f65877fe640 4 set_mon_vals no callback set 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6597579640 10 monclient: _renew_subs 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6597579640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.027+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f6590136840 con 0x7f6590137940 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.028+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f6588003da0 con 0x7f6590137940 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.028+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f65880459c0 con 0x7f6590137940 2024-05-08T15:55:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.028+0000 7f6597579640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.028+0000 7f6597579640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.028+0000 7f6597579640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.028+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65900a2260 con 0x7f6590137940 2024-05-08T15:55:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.028+0000 7f6595315640 10 monclient: get_auth_request con 0x7f657403a430 auth_method 0 2024-05-08T15:55:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.033+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f65900a2260 con 0x7f6590137940 2024-05-08T15:55:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.033+0000 7f6585ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.033+0000 7f6585ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:42.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.285+0000 7f6597579640 10 start_mon_command cmd=[{"prefix": "quorum_status"}] 2024-05-08T15:55:42.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.285+0000 7f6597579640 10 monclient: _send_command 2 [{"prefix": "quorum_status"}] 2024-05-08T15:55:42.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.285+0000 7f6597579640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.285+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f65900753d0 con 0x7f6590137940 2024-05-08T15:55:42.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.286+0000 7f6585ffb640 1 -- 172.21.15.7:0/2015355891 <== mon.4 v1:172.21.15.7:6791/0 10 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+2702 (unknown 254746657 0 4079628676) 0x7f6588014b50 con 0x7f6590137940 2024-05-08T15:55:42.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.286+0000 7f6585ffb640 10 monclient: handle_mon_command_ack 2 [{"prefix": "quorum_status"}] 2024-05-08T15:55:42.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.286+0000 7f6585ffb640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:42.282 INFO:teuthology.orchestra.run.smithi007.stdout:{"election_epoch":60,"quorum":[0,1,2,3,4,5,6,7,8],"quorum_names":["a","f","b","g","c","h","d","i","e"],"quorum_leader_name":"f","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","modified":"2024-05-08T15:46:23.145395Z","created":"2024-05-08T15:46:23.145395Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6789","nonce":0}]},"addr":"172.21.15.7:6789/0","public_addr":"172.21.15.7:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"f","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6790","nonce":0}]},"addr":"172.21.15.7:6790/0","public_addr":"172.21.15.7:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":3,"name":"g","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6790","nonce":0}]},"addr":"172.21.15.81:6790/0","public_addr":"172.21.15.81:6790/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":4,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6791","nonce":0}]},"addr":"172.21.15.7:6791/0","public_addr":"172.21.15.7:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":5,"name":"h","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6791","nonce":0}]},"addr":"172.21.15.81:6791/0","public_addr":"172.21.15.81:6791/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":6,"name":"d","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6792","nonce":0}]},"addr":"172.21.15.7:6792/0","public_addr":"172.21.15.7:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":7,"name":"i","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6792","nonce":0}]},"addr":"172.21.15.81:6792/0","public_addr":"172.21.15.81:6792/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":8,"name":"e","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6793","nonce":0}]},"addr":"172.21.15.7:6793/0","public_addr":"172.21.15.7:6793/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T15:55:42.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 >> v1:172.21.15.81:6812/847627118 conn(0x7f657403a430 legacy=0x7f657403c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 10 monclient: shutdown 2024-05-08T15:55:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 >> v1:172.21.15.7:6791/0 conn(0x7f6590137940 legacy=0x7f65901d4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 shutdown_connections 2024-05-08T15:55:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 >> 172.21.15.7:0/2015355891 conn(0x7f65900a9fa0 msgr2=0x7f6590130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 shutdown_connections 2024-05-08T15:55:42.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.288+0000 7f6597579640 1 -- 172.21.15.7:0/2015355891 wait complete. 2024-05-08T15:55:42.294 INFO:tasks.mon_thrash.ceph_manager:quorum is size 9 2024-05-08T15:55:42.294 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T15:55:42.308 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T15:55:42.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:55:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.417+0000 7f833a3f4640 10 monclient: get_monmap_and_config 2024-05-08T15:55:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.417+0000 7f833a3f4640 10 monclient: build_initial_monmap 2024-05-08T15:55:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.417+0000 7f833a3f4640 10 monclient: monmap: 2024-05-08T15:55:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.413 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:42.418485+0000 2024-05-08T15:55:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:42.418485+0000 2024-05-08T15:55:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:42.414 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:42.415 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:42.415 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 1 Processor -- start 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 1 -- start start 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient: init 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient: _add_conns ranks=[5,8,2,1,3,0,4,6,7] 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient(hunting): picked mon.noname-h con 0x7f8334137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient(hunting): picked mon.noname-e con 0x7f83340a49a0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient(hunting): picked mon.noname-b con 0x7f83340a41b0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:42.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83340a2c70 con 0x7f83340a41b0 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83340a2f50 con 0x7f8334137940 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83340a1670 con 0x7f83340a49a0 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f833a3f4640 10 monclient(hunting): authenticate will time out at 1350.533383s 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f83337fe640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f83340a49a0 0x7f833413cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55908/0 (socket says 172.21.15.7:55908) 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f8338991640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f83340a41b0 0x7f8334140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51122/0 (socket says 172.21.15.7:51122) 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.418+0000 7f83337fe640 1 -- 172.21.15.7:0/2717734609 learned_addr learned my addr 172.21.15.7:0/2717734609 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2839840708 0 0) 0x7f83340a2c70 con 0x7f83340a41b0 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): my global_id is 96104 2024-05-08T15:55:42.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f831c003520 con 0x7f83340a41b0 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3873650685 0 0) 0x7f83340a1670 con 0x7f83340a49a0 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): my global_id is 144206 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83340a2c70 con 0x7f83340a49a0 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1703161351 0 0) 0x7f83340a2f50 con 0x7f8334137940 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient(hunting): my global_id is 6728 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83340a1670 con 0x7f8334137940 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1883115577 0 0) 0x7f831c003520 con 0x7f83340a41b0 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 >> v1:172.21.15.7:6793/0 conn(0x7f83340a49a0 legacy=0x7f833413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 >> v1:172.21.15.81:6791/0 conn(0x7f8334137940 legacy=0x7f8334137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 monclient: found mon.noname-b 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83340a39f0 con 0x7f83340a41b0 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: _finish_auth 0 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f832c001b30 con 0x7f83340a41b0 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f833a3f4640 5 monclient: authenticate success, global_id 96104 2024-05-08T15:55:42.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f833a3f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: dump: 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.429 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f833a3f4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f832c001e50 con 0x7f83340a41b0 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 1 -- 172.21.15.7:0/2717734609 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f832c0020c0 con 0x7f83340a41b0 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f833a3f4640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f833a3f4640 4 set_mon_vals no callback set 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.419+0000 7f8332ffd640 10 monclient: dump: 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.430 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.431 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 10 monclient: shutdown 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 1 -- 172.21.15.7:0/2717734609 >> v1:172.21.15.7:6790/0 conn(0x7f83340a41b0 legacy=0x7f8334140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 1 -- 172.21.15.7:0/2717734609 shutdown_connections 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 1 -- 172.21.15.7:0/2717734609 >> 172.21.15.7:0/2717734609 conn(0x7f83340a9fa0 msgr2=0x7f83340aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 1 -- 172.21.15.7:0/2717734609 shutdown_connections 2024-05-08T15:55:42.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 1 -- 172.21.15.7:0/2717734609 wait complete. 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 10 monclient: build_initial_monmap 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.420+0000 7f833a3f4640 10 monclient: monmap: 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:42.433 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 1 Processor -- start 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 1 -- start start 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient: init 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient: _add_conns ranks=[4,7,6,1,2,5,3,8,0] 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient(hunting): picked mon.noname-c con 0x7f83340a41b0 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient(hunting): picked mon.noname-i con 0x7f83340a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient(hunting): picked mon.noname-d con 0x7f8334137940 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83340a1f30 con 0x7f83340a41b0 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83340a0f60 con 0x7f8334137940 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8334135770 con 0x7f83340a49a0 2024-05-08T15:55:42.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f833a3f4640 10 monclient(hunting): authenticate will time out at 1350.536386s 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f8333fff640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f83340a41b0 0x7f8334136f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51656/0 (socket says 172.21.15.7:51656) 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.421+0000 7f8333fff640 1 -- 172.21.15.7:0/392528551 learned_addr learned my addr 172.21.15.7:0/392528551 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2671572574 0 0) 0x7f83340a1f30 con 0x7f83340a41b0 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): my global_id is 124492 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8320003520 con 0x7f83340a41b0 2024-05-08T15:55:42.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1321606506 0 0) 0x7f83340a0f60 con 0x7f8334137940 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): my global_id is 124665 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83340a1f30 con 0x7f8334137940 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533919967 0 0) 0x7f8334135770 con 0x7f83340a49a0 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): my global_id is 115306 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83340a0f60 con 0x7f83340a49a0 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1071347242 0 0) 0x7f8320003520 con 0x7f83340a41b0 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8334135770 con 0x7f83340a41b0 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8324001a30 con 0x7f83340a41b0 2024-05-08T15:55:42.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): dump: 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.437 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 715864333 0 0) 0x7f83340a1f30 con 0x7f8334137940 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f8320003520 con 0x7f8334137940 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f832c001af0 con 0x7f8334137940 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient(hunting): dump: 2024-05-08T15:55:42.438 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.439 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4263370731 0 0) 0x7f83340a0f60 con 0x7f83340a49a0 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f83340a1f30 con 0x7f83340a49a0 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 775526325 0 0) 0x7f8334135770 con 0x7f83340a41b0 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 >> v1:172.21.15.81:6792/0 conn(0x7f83340a49a0 legacy=0x7f83341d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 >> v1:172.21.15.7:6792/0 conn(0x7f8334137940 legacy=0x7f83341d4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: _finish_hunting 0 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 monclient: found mon.c 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8334133dd0 con 0x7f83340a41b0 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: _finish_auth 0 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: _check_auth_tickets 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f833a3f4640 5 monclient: authenticate success, global_id 124492 2024-05-08T15:55:42.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f833a3f4640 10 monclient: _renew_subs 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f833a3f4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6791/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f8334137690 con 0x7f83340a41b0 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f833a3f4640 10 monclient: _renew_subs 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f833a3f4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6791/0 -- mon_subscribe({osdmap=0}) -- 0x7f8334134650 con 0x7f83340a41b0 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f8324003940 con 0x7f83340a41b0 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f8324003eb0 con 0x7f83340a41b0 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f83327fc640 4 set_mon_vals no callback set 2024-05-08T15:55:42.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.422+0000 7f8330ff9640 10 monclient: dump: 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.442 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.423+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f83240040d0 con 0x7f83340a41b0 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.423+0000 7f833a3f4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.423+0000 7f833a3f4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.423+0000 7f833a3f4640 10 monclient: _send_mon_message to mon.c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.423+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.7:6791/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8314005080 con 0x7f83340a41b0 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.423+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f8324046d60 con 0x7f83340a41b0 2024-05-08T15:55:42.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.423+0000 7f83337fe640 10 monclient: get_auth_request con 0x7f832003a270 auth_method 0 2024-05-08T15:55:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.428+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mon.4 v1:172.21.15.7:6791/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f8324015f70 con 0x7f83340a41b0 2024-05-08T15:55:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.428+0000 7f8330ff9640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.428+0000 7f8330ff9640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:42.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.608+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8314000b90 con 0x7f832003a270 2024-05-08T15:55:42.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.611+0000 7f8330ff9640 1 -- 172.21.15.7:0/392528551 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33850 (unknown 2965378022 0 1458842292) 0x7f8314000b90 con 0x7f832003a270 2024-05-08T15:55:42.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:55:42.607 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:55:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 >> v1:172.21.15.81:6812/847627118 conn(0x7f832003a270 legacy=0x7f832003c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 10 monclient: shutdown 2024-05-08T15:55:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 >> v1:172.21.15.7:6791/0 conn(0x7f83340a41b0 legacy=0x7f8334136f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 shutdown_connections 2024-05-08T15:55:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 >> 172.21.15.7:0/392528551 conn(0x7f83340a9fa0 msgr2=0x7f8334130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:42.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 shutdown_connections 2024-05-08T15:55:42.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.612+0000 7f833a3f4640 1 -- 172.21.15.7:0/392528551 wait complete. 2024-05-08T15:55:42.619 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":576,"stamp":"2024-05-08T15:55:41.207599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":221728,"kb_used_data":18008,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566009312,"statfs":{"total":579820584960,"available":579593535488,"internally_reserved":0,"allocated":18440192,"data_stored":9126224,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":208525986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":617,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771856+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:54:17.771856+0000","last_peered":"2024-05-08T15:54:17.771856+0000","last_clean":"2024-05-08T15:54:17.771856+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:54:17.771856+0000","last_undegraded":"2024-05-08T15:54:17.771856+0000","last_fullsized":"2024-05-08T15:54:17.771856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:14.941756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":619,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776774+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:54:17.776774+0000","last_peered":"2024-05-08T15:54:17.776774+0000","last_clean":"2024-05-08T15:54:17.776774+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:54:17.776774+0000","last_undegraded":"2024-05-08T15:54:17.776774+0000","last_fullsized":"2024-05-08T15:54:17.776774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:28.945664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.772045+0000","last_change":"2024-05-08T15:47:11.173944+0000","last_active":"2024-05-08T15:54:17.772045+0000","last_peered":"2024-05-08T15:54:17.772045+0000","last_clean":"2024-05-08T15:54:17.772045+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:54:17.772045+0000","last_undegraded":"2024-05-08T15:54:17.772045+0000","last_fullsized":"2024-05-08T15:54:17.772045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:30.537193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":616,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.842186+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:54:17.842186+0000","last_peered":"2024-05-08T15:54:17.842186+0000","last_clean":"2024-05-08T15:54:17.842186+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:54:17.842186+0000","last_undegraded":"2024-05-08T15:54:17.842186+0000","last_fullsized":"2024-05-08T15:54:17.842186+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":623,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771787+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:54:17.771787+0000","last_peered":"2024-05-08T15:54:17.771787+0000","last_clean":"2024-05-08T15:54:17.771787+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:54:17.771787+0000","last_undegraded":"2024-05-08T15:54:17.771787+0000","last_fullsized":"2024-05-08T15:54:17.771787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":617,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776615+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:54:17.776615+0000","last_peered":"2024-05-08T15:54:17.776615+0000","last_clean":"2024-05-08T15:54:17.776615+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:54:17.776615+0000","last_undegraded":"2024-05-08T15:54:17.776615+0000","last_fullsized":"2024-05-08T15:54:17.776615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:49.677393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":626,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776210+0000","last_change":"2024-05-08T15:47:11.176829+0000","last_active":"2024-05-08T15:54:17.776210+0000","last_peered":"2024-05-08T15:54:17.776210+0000","last_clean":"2024-05-08T15:54:17.776210+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:54:17.776210+0000","last_undegraded":"2024-05-08T15:54:17.776210+0000","last_fullsized":"2024-05-08T15:54:17.776210+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:32.476618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771939+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:54:17.771939+0000","last_peered":"2024-05-08T15:54:17.771939+0000","last_clean":"2024-05-08T15:54:17.771939+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:54:17.771939+0000","last_undegraded":"2024-05-08T15:54:17.771939+0000","last_fullsized":"2024-05-08T15:54:17.771939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:30.243149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":774,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.772231+0000","last_change":"2024-05-08T15:47:04.948326+0000","last_active":"2024-05-08T15:54:17.772231+0000","last_peered":"2024-05-08T15:54:17.772231+0000","last_clean":"2024-05-08T15:54:17.772231+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:54:17.772231+0000","last_undegraded":"2024-05-08T15:54:17.772231+0000","last_fullsized":"2024-05-08T15:54:17.772231+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_clean_scrub_stamp":"2024-05-08T15:47:03.939015+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:25.024655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38324,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333516,"statfs":{"total":96636764160,"available":96597520384,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640361,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30056,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341784,"statfs":{"total":96636764160,"available":96605986816,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640362,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640363,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640362,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30004,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341836,"statfs":{"total":96636764160,"available":96606040064,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640361,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30056,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341784,"statfs":{"total":96636764160,"available":96605986816,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:55:42.620 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:55:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.727+0000 7f7cf3ec8640 10 monclient: get_monmap_and_config 2024-05-08T15:55:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.727+0000 7f7cf3ec8640 10 monclient: build_initial_monmap 2024-05-08T15:55:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.727+0000 7f7cf3ec8640 10 monclient: monmap: 2024-05-08T15:55:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:42.728623+0000 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:42.728623+0000 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:42.724 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:42.725 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:42.725 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:42.725 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:42.725 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 1 Processor -- start 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 1 -- start start 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient: init 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient: _add_conns ranks=[4,1,2,5,7,3,0,8,6] 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient(hunting): picked mon.noname-c con 0x7f7cec0a5430 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient(hunting): picked mon.noname-f con 0x7f7cec0a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient(hunting): picked mon.noname-b con 0x7f7cec13e380 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec0a2990 con 0x7f7cec0a49a0 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec0a2c70 con 0x7f7cec13e380 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec0a2f50 con 0x7f7cec0a5430 2024-05-08T15:55:42.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.728+0000 7f7cf3ec8640 10 monclient(hunting): authenticate will time out at 1350.843559s 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf1c64640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f7cec0a5430 0x7f7cec0a5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:51670/0 (socket says 172.21.15.7:51670) 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf1c64640 1 -- 172.21.15.7:0/1900760166 learned_addr learned my addr 172.21.15.7:0/1900760166 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf2465640 1 --1- 172.21.15.7:0/1900760166 >> v1:172.21.15.7:6790/0 conn(0x7f7cec13e380 0x7f7cec140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:51124/0 (socket says 172.21.15.7:51124) 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2896113805 0 0) 0x7f7cec0a2c70 con 0x7f7cec13e380 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): my global_id is 96113 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd8003520 con 0x7f7cec13e380 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352959661 0 0) 0x7f7cec0a2f50 con 0x7f7cec0a5430 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): my global_id is 124501 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cec0a2c70 con 0x7f7cec0a5430 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 957694445 0 0) 0x7f7cec0a2990 con 0x7f7cec0a49a0 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient(hunting): my global_id is 6526 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cec0a2f50 con 0x7f7cec0a49a0 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1828635516 0 0) 0x7f7cec0a2c70 con 0x7f7cec0a5430 2024-05-08T15:55:42.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 >> v1:172.21.15.7:6790/0 conn(0x7f7cec13e380 legacy=0x7f7cec140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 >> v1:172.21.15.81:6789/0 conn(0x7f7cec0a49a0 legacy=0x7f7cec13cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: _finish_hunting 0 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 monclient: found mon.noname-c 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec0a4300 con 0x7f7cec0a5430 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: _finish_auth 0 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: _check_auth_tickets 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7ce8001790 con 0x7f7cec0a5430 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf3ec8640 5 monclient: authenticate success, global_id 124501 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf3ec8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: dump: 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.736 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf3ec8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7ce80019f0 con 0x7f7cec0a5430 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 1 -- 172.21.15.7:0/1900760166 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7ce8004be0 con 0x7f7cec0a5430 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf3ec8640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf3ec8640 4 set_mon_vals no callback set 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.729+0000 7f7cf0c62640 10 monclient: dump: 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.738 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 10 monclient: shutdown 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/1900760166 >> v1:172.21.15.7:6791/0 conn(0x7f7cec0a5430 legacy=0x7f7cec0a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/1900760166 shutdown_connections 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/1900760166 >> 172.21.15.7:0/1900760166 conn(0x7f7cec0a9fa0 msgr2=0x7f7cec0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/1900760166 shutdown_connections 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/1900760166 wait complete. 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 10 monclient: build_initial_monmap 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.730+0000 7f7cf3ec8640 10 monclient: monmap: 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:42.739 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 1 Processor -- start 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 1 -- start start 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient: init 2024-05-08T15:55:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient: _add_conns ranks=[6,7,1,2,0,3,5,4,8] 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient(hunting): picked mon.noname-d con 0x7f7cec0a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient(hunting): picked mon.noname-i con 0x7f7cec0a5430 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient(hunting): picked mon.noname-f con 0x7f7cec13e380 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec137650 con 0x7f7cec13e380 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec136930 con 0x7f7cec0a49a0 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7cec135640 con 0x7f7cec0a5430 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf3ec8640 10 monclient(hunting): authenticate will time out at 1350.846459s 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf1c64640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f7cec0a49a0 0x7f7cec1c9530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35268/0 (socket says 172.21.15.7:35268) 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.731+0000 7f7cf1c64640 1 -- 172.21.15.7:0/814108636 learned_addr learned my addr 172.21.15.7:0/814108636 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 52288450 0 0) 0x7f7cec136930 con 0x7f7cec0a49a0 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): my global_id is 124674 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cd0003520 con 0x7f7cec0a49a0 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2402434261 0 0) 0x7f7cec135640 con 0x7f7cec0a5430 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): my global_id is 115315 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cec136930 con 0x7f7cec0a5430 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2479896896 0 0) 0x7f7cec137650 con 0x7f7cec13e380 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): my global_id is 6535 2024-05-08T15:55:42.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7cec135640 con 0x7f7cec13e380 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2649790854 0 0) 0x7f7cd0003520 con 0x7f7cec0a49a0 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7cec137650 con 0x7f7cec0a49a0 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7ce8004b20 con 0x7f7cec0a49a0 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.743 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 291588736 0 0) 0x7f7cec136930 con 0x7f7cec0a5430 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7cd0003520 con 0x7f7cec0a5430 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7ce0001a50 con 0x7f7cec0a5430 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 950866868 0 0) 0x7f7cec137650 con 0x7f7cec0a49a0 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 >> v1:172.21.15.81:6792/0 conn(0x7f7cec0a5430 legacy=0x7f7cec1d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 >> v1:172.21.15.81:6789/0 conn(0x7f7cec13e380 legacy=0x7f7cec1d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 monclient: found mon.d 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec137090 con 0x7f7cec0a49a0 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cf3ec8640 5 monclient: authenticate success, global_id 124674 2024-05-08T15:55:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cf3ec8640 10 monclient: _renew_subs 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cf3ec8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec1361a0 con 0x7f7cec0a49a0 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cf3ec8640 10 monclient: _renew_subs 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cf3ec8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f7cec136430 con 0x7f7cec0a49a0 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7ce80019f0 con 0x7f7cec0a49a0 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7ce8005230 con 0x7f7cec0a49a0 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cdffff640 4 set_mon_vals no callback set 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.732+0000 7f7cde7fc640 10 monclient: dump: 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.733+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7ce80105b0 con 0x7f7cec0a49a0 2024-05-08T15:55:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.733+0000 7f7cf3ec8640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.733+0000 7f7cf3ec8640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.733+0000 7f7cf3ec8640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.733+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cd4005080 con 0x7f7cec0a49a0 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.733+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f7ce8046d60 con 0x7f7cec0a49a0 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.733+0000 7f7cf1463640 10 monclient: get_auth_request con 0x7f7cd0039f30 auth_method 0 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.738+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f7ce8016090 con 0x7f7cec0a49a0 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.738+0000 7f7cde7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.738+0000 7f7cde7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:42.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.921+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7cd4000b90 con 0x7f7cd0039f30 2024-05-08T15:55:42.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.923+0000 7f7cde7fc640 1 -- 172.21.15.7:0/814108636 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33850 (unknown 2965378022 0 1458842292) 0x7f7cd4000b90 con 0x7f7cd0039f30 2024-05-08T15:55:42.919 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:55:42.919 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 >> v1:172.21.15.81:6812/847627118 conn(0x7f7cd0039f30 legacy=0x7f7cd003c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 10 monclient: shutdown 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 >> v1:172.21.15.7:6792/0 conn(0x7f7cec0a49a0 legacy=0x7f7cec1c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 shutdown_connections 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 >> 172.21.15.7:0/814108636 conn(0x7f7cec0a9fa0 msgr2=0x7f7cec140cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 shutdown_connections 2024-05-08T15:55:42.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:42.925+0000 7f7cf3ec8640 1 -- 172.21.15.7:0/814108636 wait complete. 2024-05-08T15:55:42.932 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":576,"stamp":"2024-05-08T15:55:41.207599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":221728,"kb_used_data":18008,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566009312,"statfs":{"total":579820584960,"available":579593535488,"internally_reserved":0,"allocated":18440192,"data_stored":9126224,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":208525986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002406"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":617,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771856+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:54:17.771856+0000","last_peered":"2024-05-08T15:54:17.771856+0000","last_clean":"2024-05-08T15:54:17.771856+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:54:17.771856+0000","last_undegraded":"2024-05-08T15:54:17.771856+0000","last_fullsized":"2024-05-08T15:54:17.771856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:14.941756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":619,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776774+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:54:17.776774+0000","last_peered":"2024-05-08T15:54:17.776774+0000","last_clean":"2024-05-08T15:54:17.776774+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:54:17.776774+0000","last_undegraded":"2024-05-08T15:54:17.776774+0000","last_fullsized":"2024-05-08T15:54:17.776774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:28.945664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.772045+0000","last_change":"2024-05-08T15:47:11.173944+0000","last_active":"2024-05-08T15:54:17.772045+0000","last_peered":"2024-05-08T15:54:17.772045+0000","last_clean":"2024-05-08T15:54:17.772045+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:54:17.772045+0000","last_undegraded":"2024-05-08T15:54:17.772045+0000","last_fullsized":"2024-05-08T15:54:17.772045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:30.537193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":616,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.842186+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:54:17.842186+0000","last_peered":"2024-05-08T15:54:17.842186+0000","last_clean":"2024-05-08T15:54:17.842186+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:54:17.842186+0000","last_undegraded":"2024-05-08T15:54:17.842186+0000","last_fullsized":"2024-05-08T15:54:17.842186+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":623,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771787+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:54:17.771787+0000","last_peered":"2024-05-08T15:54:17.771787+0000","last_clean":"2024-05-08T15:54:17.771787+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:54:17.771787+0000","last_undegraded":"2024-05-08T15:54:17.771787+0000","last_fullsized":"2024-05-08T15:54:17.771787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":617,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776615+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:54:17.776615+0000","last_peered":"2024-05-08T15:54:17.776615+0000","last_clean":"2024-05-08T15:54:17.776615+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:54:17.776615+0000","last_undegraded":"2024-05-08T15:54:17.776615+0000","last_fullsized":"2024-05-08T15:54:17.776615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:49.677393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":626,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776210+0000","last_change":"2024-05-08T15:47:11.176829+0000","last_active":"2024-05-08T15:54:17.776210+0000","last_peered":"2024-05-08T15:54:17.776210+0000","last_clean":"2024-05-08T15:54:17.776210+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:54:17.776210+0000","last_undegraded":"2024-05-08T15:54:17.776210+0000","last_fullsized":"2024-05-08T15:54:17.776210+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:32.476618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771939+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:54:17.771939+0000","last_peered":"2024-05-08T15:54:17.771939+0000","last_clean":"2024-05-08T15:54:17.771939+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:54:17.771939+0000","last_undegraded":"2024-05-08T15:54:17.771939+0000","last_fullsized":"2024-05-08T15:54:17.771939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:30.243149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":774,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.772231+0000","last_change":"2024-05-08T15:47:04.948326+0000","last_active":"2024-05-08T15:54:17.772231+0000","last_peered":"2024-05-08T15:54:17.772231+0000","last_clean":"2024-05-08T15:54:17.772231+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:54:17.772231+0000","last_undegraded":"2024-05-08T15:54:17.772231+0000","last_fullsized":"2024-05-08T15:54:17.772231+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_clean_scrub_stamp":"2024-05-08T15:47:03.939015+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:25.024655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38324,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333516,"statfs":{"total":96636764160,"available":96597520384,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640361,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30056,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341784,"statfs":{"total":96636764160,"available":96605986816,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640362,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640363,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640362,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30004,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341836,"statfs":{"total":96636764160,"available":96606040064,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640361,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30056,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341784,"statfs":{"total":96636764160,"available":96605986816,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:55:42.934 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T15:55:42.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:55:43.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.040+0000 7f548b64c640 10 monclient: get_monmap_and_config 2024-05-08T15:55:43.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.040+0000 7f548b64c640 10 monclient: build_initial_monmap 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.041+0000 7f548b64c640 10 monclient: monmap: 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:43.041968+0000 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:43.041968+0000 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:43.037 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:43.038 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:43.038 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.041+0000 7f548b64c640 1 Processor -- start 2024-05-08T15:55:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.041+0000 7f548b64c640 1 -- start start 2024-05-08T15:55:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.041+0000 7f548b64c640 10 monclient: init 2024-05-08T15:55:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 10 monclient: _add_conns ranks=[0,1,8,2,7,3,4,5,6] 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 10 monclient(hunting): picked mon.noname-a con 0x7f54840a5430 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 10 monclient(hunting): picked mon.noname-f con 0x7f54840a49a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 10 monclient(hunting): picked mon.noname-e con 0x7f548413e380 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54840a2990 con 0x7f54840a5430 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54840a2c70 con 0x7f54840a49a0 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54840a2f50 con 0x7f548413e380 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f548b64c640 10 monclient(hunting): authenticate will time out at 1351.156986s 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f54893e8640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54840a5430 0x7f54840a5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34052/0 (socket says 172.21.15.7:34052) 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f54893e8640 1 -- 172.21.15.7:0/4275503149 learned_addr learned my addr 172.21.15.7:0/4275503149 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 547243017 0 0) 0x7f54840a2990 con 0x7f54840a5430 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): my global_id is 95895 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5470003520 con 0x7f54840a5430 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3158459659 0 0) 0x7f54840a2f50 con 0x7f548413e380 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): my global_id is 144215 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54840a2990 con 0x7f548413e380 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2786530686 0 0) 0x7f54840a2c70 con 0x7f54840a49a0 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 10 monclient(hunting): my global_id is 6544 2024-05-08T15:55:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54840a2f50 con 0x7f54840a49a0 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3821606748 0 0) 0x7f5470003520 con 0x7f54840a5430 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.042+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 >> v1:172.21.15.7:6793/0 conn(0x7f548413e380 legacy=0x7f5484140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 >> v1:172.21.15.81:6789/0 conn(0x7f54840a49a0 legacy=0x7f548413cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 1 monclient: found mon.noname-a 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54840a4300 con 0x7f54840a5430 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: _finish_auth 0 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5480001b30 con 0x7f54840a5430 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 5 monclient: authenticate success, global_id 95895 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: dump: 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5480001e50 con 0x7f54840a5430 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 1 -- 172.21.15.7:0/4275503149 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5480004f60 con 0x7f54840a5430 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f547bfff640 10 monclient: dump: 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.050 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 4 set_mon_vals no callback set 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 10 monclient: shutdown 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 1 -- 172.21.15.7:0/4275503149 >> v1:172.21.15.7:6789/0 conn(0x7f54840a5430 legacy=0x7f54840a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 1 -- 172.21.15.7:0/4275503149 shutdown_connections 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 1 -- 172.21.15.7:0/4275503149 >> 172.21.15.7:0/4275503149 conn(0x7f54840a9fa0 msgr2=0x7f54840aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 1 -- 172.21.15.7:0/4275503149 shutdown_connections 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.043+0000 7f548b64c640 1 -- 172.21.15.7:0/4275503149 wait complete. 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.044+0000 7f548b64c640 10 monclient: build_initial_monmap 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.044+0000 7f548b64c640 10 monclient: monmap: 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:43.052 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.044+0000 7f548b64c640 1 Processor -- start 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.044+0000 7f548b64c640 1 -- start start 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.044+0000 7f548b64c640 10 monclient: init 2024-05-08T15:55:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 10 monclient: _add_conns ranks=[2,0,5,7,6,3,1,4,8] 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 10 monclient(hunting): picked mon.noname-b con 0x7f54840a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 10 monclient(hunting): picked mon.noname-a con 0x7f54840a5430 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 10 monclient(hunting): picked mon.noname-h con 0x7f548413e380 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54840a0f60 con 0x7f54840a5430 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f548409d130 con 0x7f54840a49a0 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f548409e2d0 con 0x7f548413e380 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f548b64c640 10 monclient(hunting): authenticate will time out at 1351.160006s 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5488be7640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f54840a5430 0x7f54841d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34060/0 (socket says 172.21.15.7:34060) 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5488be7640 1 -- 172.21.15.7:0/861530884 learned_addr learned my addr 172.21.15.7:0/861530884 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2167373984 0 0) 0x7f54840a0f60 con 0x7f54840a5430 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): my global_id is 95904 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5468003520 con 0x7f54840a5430 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3658490480 0 0) 0x7f548409d130 con 0x7f54840a49a0 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): my global_id is 96122 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54840a0f60 con 0x7f54840a49a0 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1743648812 0 0) 0x7f5468003520 con 0x7f54840a5430 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f548409d130 con 0x7f54840a5430 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5474001a30 con 0x7f54840a5430 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:43.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.056 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.045+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4168682625 0 0) 0x7f54840a0f60 con 0x7f54840a49a0 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5468003520 con 0x7f54840a49a0 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 968831746 0 0) 0x7f548409e2d0 con 0x7f548413e380 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient(hunting): my global_id is 6737 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54840a0f60 con 0x7f548413e380 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5480004ea0 con 0x7f54840a49a0 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.057 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2821777713 0 0) 0x7f548409d130 con 0x7f54840a5430 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 >> v1:172.21.15.81:6791/0 conn(0x7f548413e380 legacy=0x7f54841d50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 >> v1:172.21.15.7:6790/0 conn(0x7f54840a49a0 legacy=0x7f54840a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 monclient: found mon.a 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54841c9b30 con 0x7f54840a5430 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: _finish_auth 0 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f548b64c640 5 monclient: authenticate success, global_id 95904 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f548b64c640 10 monclient: _renew_subs 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f548b64c640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f54840a07b0 con 0x7f54840a5430 2024-05-08T15:55:43.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f54740038c0 con 0x7f54840a5430 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5474003e30 con 0x7f54840a5430 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f5479ffb640 10 monclient: dump: 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.060 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f547b7fe640 4 set_mon_vals no callback set 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f548b64c640 10 monclient: _renew_subs 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f548b64c640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.046+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f548409d5c0 con 0x7f54840a5430 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.047+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f5474004050 con 0x7f54840a5430 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.047+0000 7f548b64c640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.047+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f5474046e60 con 0x7f54840a5430 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.047+0000 7f548b64c640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.047+0000 7f548b64c640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.047+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54840753d0 con 0x7f54840a5430 2024-05-08T15:55:43.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.047+0000 7f54893e8640 10 monclient: get_auth_request con 0x7f546803a230 auth_method 0 2024-05-08T15:55:43.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.052+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f54740752d0 con 0x7f54840a5430 2024-05-08T15:55:43.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.052+0000 7f5479ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:43.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.052+0000 7f5479ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:43.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.230+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54840a1f30 con 0x7f546803a230 2024-05-08T15:55:43.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.232+0000 7f5479ffb640 1 -- 172.21.15.7:0/861530884 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33850 (unknown 2965378022 0 3746659847) 0x7f54840a1f30 con 0x7f546803a230 2024-05-08T15:55:43.228 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:55:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 >> v1:172.21.15.81:6812/847627118 conn(0x7f546803a230 legacy=0x7f546803c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 10 monclient: shutdown 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 >> v1:172.21.15.7:6789/0 conn(0x7f54840a5430 legacy=0x7f54841d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 shutdown_connections 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 >> 172.21.15.7:0/861530884 conn(0x7f54840a9fa0 msgr2=0x7f5484140f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 shutdown_connections 2024-05-08T15:55:43.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.234+0000 7f548b64c640 1 -- 172.21.15.7:0/861530884 wait complete. 2024-05-08T15:55:43.240 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":577,"stamp":"2024-05-08T15:55:43.207934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":221728,"kb_used_data":18008,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566009312,"statfs":{"total":579820584960,"available":579593535488,"internally_reserved":0,"allocated":18440192,"data_stored":9126224,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":208525986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002396"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":617,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771856+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:54:17.771856+0000","last_peered":"2024-05-08T15:54:17.771856+0000","last_clean":"2024-05-08T15:54:17.771856+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:54:17.771856+0000","last_undegraded":"2024-05-08T15:54:17.771856+0000","last_fullsized":"2024-05-08T15:54:17.771856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:45:14.941756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":619,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776774+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:54:17.776774+0000","last_peered":"2024-05-08T15:54:17.776774+0000","last_clean":"2024-05-08T15:54:17.776774+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:54:17.776774+0000","last_undegraded":"2024-05-08T15:54:17.776774+0000","last_fullsized":"2024-05-08T15:54:17.776774+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:28.945664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.772045+0000","last_change":"2024-05-08T15:47:11.173944+0000","last_active":"2024-05-08T15:54:17.772045+0000","last_peered":"2024-05-08T15:54:17.772045+0000","last_clean":"2024-05-08T15:54:17.772045+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:54:17.772045+0000","last_undegraded":"2024-05-08T15:54:17.772045+0000","last_fullsized":"2024-05-08T15:54:17.772045+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:30.537193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":616,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.842186+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:54:17.842186+0000","last_peered":"2024-05-08T15:54:17.842186+0000","last_clean":"2024-05-08T15:54:17.842186+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:54:17.842186+0000","last_undegraded":"2024-05-08T15:54:17.842186+0000","last_fullsized":"2024-05-08T15:54:17.842186+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":623,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771787+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:54:17.771787+0000","last_peered":"2024-05-08T15:54:17.771787+0000","last_clean":"2024-05-08T15:54:17.771787+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:54:17.771787+0000","last_undegraded":"2024-05-08T15:54:17.771787+0000","last_fullsized":"2024-05-08T15:54:17.771787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":617,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776615+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:54:17.776615+0000","last_peered":"2024-05-08T15:54:17.776615+0000","last_clean":"2024-05-08T15:54:17.776615+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:54:17.776615+0000","last_undegraded":"2024-05-08T15:54:17.776615+0000","last_fullsized":"2024-05-08T15:54:17.776615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:49.677393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":626,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776210+0000","last_change":"2024-05-08T15:47:11.176829+0000","last_active":"2024-05-08T15:54:17.776210+0000","last_peered":"2024-05-08T15:54:17.776210+0000","last_clean":"2024-05-08T15:54:17.776210+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:54:17.776210+0000","last_undegraded":"2024-05-08T15:54:17.776210+0000","last_fullsized":"2024-05-08T15:54:17.776210+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:14:32.476618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771939+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:54:17.771939+0000","last_peered":"2024-05-08T15:54:17.771939+0000","last_clean":"2024-05-08T15:54:17.771939+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:54:17.771939+0000","last_undegraded":"2024-05-08T15:54:17.771939+0000","last_fullsized":"2024-05-08T15:54:17.771939+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:30.243149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":774,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.772231+0000","last_change":"2024-05-08T15:47:04.948326+0000","last_active":"2024-05-08T15:54:17.772231+0000","last_peered":"2024-05-08T15:54:17.772231+0000","last_clean":"2024-05-08T15:54:17.772231+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:54:17.772231+0000","last_undegraded":"2024-05-08T15:54:17.772231+0000","last_fullsized":"2024-05-08T15:54:17.772231+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:03.939015+0000","last_clean_scrub_stamp":"2024-05-08T15:47:03.939015+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:15:25.024655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38324,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333516,"statfs":{"total":96636764160,"available":96597520384,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640362,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30056,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341784,"statfs":{"total":96636764160,"available":96605986816,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640363,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640364,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640363,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30004,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341836,"statfs":{"total":96636764160,"available":96606040064,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640362,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30056,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341784,"statfs":{"total":96636764160,"available":96605986816,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:55:43.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T15:55:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.347+0000 7f052466d640 10 monclient: get_monmap_and_config 2024-05-08T15:55:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.347+0000 7f052466d640 10 monclient: build_initial_monmap 2024-05-08T15:55:43.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.347+0000 7f052466d640 10 monclient: monmap: 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:43.348912+0000 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:43.348912+0000 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:43.344 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:43.345 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:43.345 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.348+0000 7f052466d640 1 Processor -- start 2024-05-08T15:55:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.348+0000 7f052466d640 1 -- start start 2024-05-08T15:55:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.348+0000 7f052466d640 10 monclient: init 2024-05-08T15:55:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 10 monclient: _add_conns ranks=[3,6,7,8,1,5,2,0,4] 2024-05-08T15:55:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 10 monclient(hunting): picked mon.noname-g con 0x7f051c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 10 monclient(hunting): picked mon.noname-d con 0x7f051c0a49a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 10 monclient(hunting): picked mon.noname-i con 0x7f051c0a41b0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0a2c70 con 0x7f051c137940 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0a2f50 con 0x7f051c0a49a0 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0a1670 con 0x7f051c0a41b0 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 10 monclient(hunting): authenticate will time out at 1351.463864s 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521c08640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f051c0a49a0 0x7f051c13cee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:35282/0 (socket says 172.21.15.7:35282) 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521c08640 1 -- 172.21.15.7:0/682261997 learned_addr learned my addr 172.21.15.7:0/682261997 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3094833092 0 0) 0x7f051c0a2f50 con 0x7f051c0a49a0 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient(hunting): my global_id is 124683 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0508003520 con 0x7f051c0a49a0 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1397497623 0 0) 0x7f0508003520 con 0x7f051c0a49a0 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 >> v1:172.21.15.81:6792/0 conn(0x7f051c0a41b0 legacy=0x7f051c140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 >> v1:172.21.15.81:6790/0 conn(0x7f051c137940 legacy=0x7f051c137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient: _finish_hunting 0 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 monclient: found mon.noname-d 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c0a39f0 con 0x7f051c0a49a0 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient: _finish_auth 0 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient: _check_auth_tickets 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f050c0017b0 con 0x7f051c0a49a0 2024-05-08T15:55:43.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 10 monclient: dump: 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.354 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 5 monclient: authenticate success, global_id 124683 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f052466d640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.349+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f050c001b10 con 0x7f051c0a49a0 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f0521407640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f0521407640 1 -- 172.21.15.7:0/682261997 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f050c004be0 con 0x7f051c0a49a0 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 4 set_mon_vals no callback set 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f0521407640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f0521407640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f0521407640 10 monclient: dump: 2024-05-08T15:55:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.356 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 10 monclient: shutdown 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 1 -- 172.21.15.7:0/682261997 >> v1:172.21.15.7:6792/0 conn(0x7f051c0a49a0 legacy=0x7f051c13cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 1 -- 172.21.15.7:0/682261997 shutdown_connections 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 1 -- 172.21.15.7:0/682261997 >> 172.21.15.7:0/682261997 conn(0x7f051c0a9fa0 msgr2=0x7f051c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 1 -- 172.21.15.7:0/682261997 shutdown_connections 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.350+0000 7f052466d640 1 -- 172.21.15.7:0/682261997 wait complete. 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient: build_initial_monmap 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient: monmap: 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 1 Processor -- start 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 1 -- start start 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient: init 2024-05-08T15:55:43.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient: _add_conns ranks=[1,0,3,2,5,8,4,7,6] 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient(hunting): picked mon.noname-f con 0x7f051c0a41b0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient(hunting): picked mon.noname-a con 0x7f051c0a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient(hunting): picked mon.noname-g con 0x7f051c137940 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c1399e0 con 0x7f051c0a49a0 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0a1f30 con 0x7f051c0a41b0 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f051c0a0f60 con 0x7f051c137940 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.351+0000 7f052466d640 10 monclient(hunting): authenticate will time out at 1351.466592s 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0521c08640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f051c0a49a0 0x7f051c1d1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:34072/0 (socket says 172.21.15.7:34072) 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0521c08640 1 -- 172.21.15.7:0/1092121218 learned_addr learned my addr 172.21.15.7:0/1092121218 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660003324 0 0) 0x7f051c1399e0 con 0x7f051c0a49a0 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): my global_id is 95913 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0500003550 con 0x7f051c0a49a0 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3835015983 0 0) 0x7f051c0a0f60 con 0x7f051c137940 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): my global_id is 76971 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f051c1399e0 con 0x7f051c137940 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477691639 0 0) 0x7f051c0a1f30 con 0x7f051c0a41b0 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): my global_id is 6553 2024-05-08T15:55:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f051c0a0f60 con 0x7f051c0a41b0 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 4221282313 0 0) 0x7f0500003550 con 0x7f051c0a49a0 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f051c0a1f30 con 0x7f051c0a49a0 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f050c001770 con 0x7f051c0a49a0 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1901184696 0 0) 0x7f051c1399e0 con 0x7f051c137940 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0500003550 con 0x7f051c137940 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0514001990 con 0x7f051c137940 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:43.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1516954426 0 0) 0x7f051c0a0f60 con 0x7f051c0a41b0 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f051c1399e0 con 0x7f051c0a41b0 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0518001c20 con 0x7f051c0a41b0 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2501750245 0 0) 0x7f051c0a1f30 con 0x7f051c0a49a0 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 >> v1:172.21.15.81:6790/0 conn(0x7f051c137940 legacy=0x7f051c1d4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 >> v1:172.21.15.81:6789/0 conn(0x7f051c0a41b0 legacy=0x7f051c140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:43.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 monclient: found mon.a 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c139700 con 0x7f051c0a49a0 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient: _finish_auth 0 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f0512ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f052466d640 5 monclient: authenticate success, global_id 95913 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f052466d640 10 monclient: _renew_subs 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f052466d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f051c1cc3b0 con 0x7f051c0a49a0 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f052466d640 10 monclient: _renew_subs 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f052466d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.352+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f051c141840 con 0x7f051c0a49a0 2024-05-08T15:55:43.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f050c001b10 con 0x7f051c0a49a0 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0512ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f050c0051a0 con 0x7f051c0a49a0 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0512ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0520c06640 4 set_mon_vals no callback set 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0512ffd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0512ffd640 10 monclient: dump: 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f052466d640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f050c010520 con 0x7f051c0a49a0 2024-05-08T15:55:43.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f052466d640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f052466d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.353+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0504005080 con 0x7f051c0a49a0 2024-05-08T15:55:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.354+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f050c046df0 con 0x7f051c0a49a0 2024-05-08T15:55:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.354+0000 7f0522409640 10 monclient: get_auth_request con 0x7f050003a450 auth_method 0 2024-05-08T15:55:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.358+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f050c016040 con 0x7f051c0a49a0 2024-05-08T15:55:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.358+0000 7f0512ffd640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:43.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.358+0000 7f0512ffd640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:43.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.544+0000 7f052466d640 10 start_mon_command cmd=[{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:55:43.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.544+0000 7f052466d640 10 monclient: _send_command 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:55:43.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.544+0000 7f052466d640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:43.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.544+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0504005370 con 0x7f051c0a49a0 2024-05-08T15:55:43.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.545+0000 7f0512ffd640 1 -- 172.21.15.7:0/1092121218 <== mon.0 v1:172.21.15.7:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v323) ==== 74+0+10744 (unknown 952665033 0 3493910518) 0x7f050c015940 con 0x7f051c0a49a0 2024-05-08T15:55:43.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.545+0000 7f0512ffd640 10 monclient: handle_mon_command_ack 2 [{"prefix": "osd dump", "format": "json"}] 2024-05-08T15:55:43.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.545+0000 7f0512ffd640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:55:43.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:55:43.541 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":323,"fsid":"402a637b-159c-4d0b-9f9c-7ff72b154b33","created":"2024-05-08T15:46:52.909548+0000","modified":"2024-05-08T15:54:17.765143+0000","last_up_change":"2024-05-08T15:47:03.939015+0000","last_in_change":"2024-05-08T15:46:54.573248+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":100,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T15:47:03.139226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6.059999942779541,"score_stable":6.059999942779541,"optimal_score":0.33000001311302185,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T15:47:09.761977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8f2a2966-1786-4529-bf11-c65c304769c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":320,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6804","nonce":3231689356}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6805","nonce":3231689356}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6807","nonce":3231689356}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6806","nonce":3231689356}]},"public_addr":"172.21.15.7:6804/3231689356","cluster_addr":"172.21.15.7:6805/3231689356","heartbeat_back_addr":"172.21.15.7:6807/3231689356","heartbeat_front_addr":"172.21.15.7:6806/3231689356","state":["exists","up"]},{"osd":1,"uuid":"68812ea6-7e31-48c1-a823-1748f6eda800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":320,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6800","nonce":601836183}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6801","nonce":601836183}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6803","nonce":601836183}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6802","nonce":601836183}]},"public_addr":"172.21.15.7:6800/601836183","cluster_addr":"172.21.15.7:6801/601836183","heartbeat_back_addr":"172.21.15.7:6803/601836183","heartbeat_front_addr":"172.21.15.7:6802/601836183","state":["exists","up"]},{"osd":2,"uuid":"3ca973c1-ee0b-43e5-a644-a8e110676b39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":320,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6808","nonce":2598303896}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6809","nonce":2598303896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6811","nonce":2598303896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.7:6810","nonce":2598303896}]},"public_addr":"172.21.15.7:6808/2598303896","cluster_addr":"172.21.15.7:6809/2598303896","heartbeat_back_addr":"172.21.15.7:6811/2598303896","heartbeat_front_addr":"172.21.15.7:6810/2598303896","state":["exists","up"]},{"osd":3,"uuid":"0420b958-6a66-413a-bb95-fcc848bf9708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":320,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6802","nonce":1326537731}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6805","nonce":1326537731}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6807","nonce":1326537731}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6806","nonce":1326537731}]},"public_addr":"172.21.15.81:6802/1326537731","cluster_addr":"172.21.15.81:6805/1326537731","heartbeat_back_addr":"172.21.15.81:6807/1326537731","heartbeat_front_addr":"172.21.15.81:6806/1326537731","state":["exists","up"]},{"osd":4,"uuid":"64485986-c4f5-4899-8a0c-604433e57b46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":320,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6808","nonce":193838907}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6809","nonce":193838907}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6811","nonce":193838907}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6810","nonce":193838907}]},"public_addr":"172.21.15.81:6808/193838907","cluster_addr":"172.21.15.81:6809/193838907","heartbeat_back_addr":"172.21.15.81:6811/193838907","heartbeat_front_addr":"172.21.15.81:6810/193838907","state":["exists","up"]},{"osd":5,"uuid":"9a6e6150-5226-49ae-8598-7cf36b0f42d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":320,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6800","nonce":385867939}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6801","nonce":385867939}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6804","nonce":385867939}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.81:6803","nonce":385867939}]},"public_addr":"172.21.15.81:6800/385867939","cluster_addr":"172.21.15.81:6801/385867939","heartbeat_back_addr":"172.21.15.81:6804/385867939","heartbeat_front_addr":"172.21.15.81:6803/385867939","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.702217+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.485746+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.746567+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.749309+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.745027+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T15:46:59.779789+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T15:55:43.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.546+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 >> v1:172.21.15.81:6812/847627118 conn(0x7f050003a450 legacy=0x7f050003c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.546+0000 7f052466d640 10 monclient: shutdown 2024-05-08T15:55:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.546+0000 7f052466d640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.546+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 >> v1:172.21.15.7:6789/0 conn(0x7f051c0a49a0 legacy=0x7f051c1d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.547+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 shutdown_connections 2024-05-08T15:55:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.547+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 >> 172.21.15.7:0/1092121218 conn(0x7f051c0a9fa0 msgr2=0x7f051c0a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.547+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 shutdown_connections 2024-05-08T15:55:43.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:43.547+0000 7f052466d640 1 -- 172.21.15.7:0/1092121218 wait complete. 2024-05-08T15:55:44.554 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T15:55:44.554 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T15:55:44.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.679+0000 7f9b1dd02640 10 monclient: get_monmap_and_config 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.679+0000 7f9b1dd02640 10 monclient: build_initial_monmap 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.679+0000 7f9b1dd02640 10 monclient: monmap: 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:44.680146+0000 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:44.680146+0000 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:44.675 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:44.676 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:44.676 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.679+0000 7f9b1dd02640 1 Processor -- start 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 1 -- start start 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient: init 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient: _add_conns ranks=[8,4,3,0,1,5,2,7,6] 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient(hunting): picked mon.noname-e con 0x7f9b18137900 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient(hunting): picked mon.noname-c con 0x7f9b180a4110 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient(hunting): picked mon.noname-g con 0x7f9b181383f0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b180a48c0 con 0x7f9b181383f0 2024-05-08T15:55:44.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b180a28a0 con 0x7f9b180a4110 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b180a2b80 con 0x7f9b18137900 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b1dd02640 10 monclient(hunting): authenticate will time out at 1352.795167s 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b16ffd640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f9b180a4110 0x7f9b1813ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:44972/0 (socket says 172.21.15.7:44972) 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b16ffd640 1 -- 172.21.15.7:0/2561591651 learned_addr learned my addr 172.21.15.7:0/2561591651 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b177fe640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f9b18137900 0x7f9b18137ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55770/0 (socket says 172.21.15.7:55770) 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1614659265 0 0) 0x7f9b180a28a0 con 0x7f9b180a4110 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): my global_id is 124510 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b00003520 con 0x7f9b180a4110 2024-05-08T15:55:44.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 137974060 0 0) 0x7f9b180a2b80 con 0x7f9b18137900 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.680+0000 7f9b167fc640 10 monclient(hunting): my global_id is 144224 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b180a28a0 con 0x7f9b18137900 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 485257322 0 0) 0x7f9b00003520 con 0x7f9b180a4110 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 >> v1:172.21.15.7:6793/0 conn(0x7f9b18137900 legacy=0x7f9b18137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 >> v1:172.21.15.81:6790/0 conn(0x7f9b181383f0 legacy=0x7f9b18140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 monclient: found mon.noname-c 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b180a2e60 con 0x7f9b180a4110 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:44.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 5 monclient: authenticate success, global_id 124510 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9b0c0017b0 con 0x7f9b180a4110 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: dump: 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.686 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9b0c001a20 con 0x7f9b180a4110 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 1 -- 172.21.15.7:0/2561591651 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9b0c004be0 con 0x7f9b180a4110 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 4 set_mon_vals no callback set 2024-05-08T15:55:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b167fc640 10 monclient: dump: 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 10 monclient: shutdown 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 1 -- 172.21.15.7:0/2561591651 >> v1:172.21.15.7:6791/0 conn(0x7f9b180a4110 legacy=0x7f9b1813ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 1 -- 172.21.15.7:0/2561591651 shutdown_connections 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 1 -- 172.21.15.7:0/2561591651 >> 172.21.15.7:0/2561591651 conn(0x7f9b180a9ec0 msgr2=0x7f9b180aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 1 -- 172.21.15.7:0/2561591651 shutdown_connections 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.681+0000 7f9b1dd02640 1 -- 172.21.15.7:0/2561591651 wait complete. 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.682+0000 7f9b1dd02640 10 monclient: build_initial_monmap 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.682+0000 7f9b1dd02640 10 monclient: monmap: 2024-05-08T15:55:44.689 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.682+0000 7f9b1dd02640 1 Processor -- start 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 1 -- start start 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient: init 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient: _add_conns ranks=[2,3,1,6,8,0,7,4,5] 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient(hunting): picked mon.noname-b con 0x7f9b18137900 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient(hunting): picked mon.noname-g con 0x7f9b181383f0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient(hunting): picked mon.noname-f con 0x7f9b181d2f00 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b181339e0 con 0x7f9b181d2f00 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b18132d20 con 0x7f9b18137900 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b181309f0 con 0x7f9b181383f0 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b1dd02640 10 monclient(hunting): authenticate will time out at 1352.798147s 2024-05-08T15:55:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b177fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f9b18137900 0x7f9b1812fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37028/0 (socket says 172.21.15.7:37028) 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9b177fe640 1 -- 172.21.15.7:0/3117893446 learned_addr learned my addr 172.21.15.7:0/3117893446 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3281573589 0 0) 0x7f9b18132d20 con 0x7f9b18137900 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9afffff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9afffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9afffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9afffff640 10 monclient(hunting): my global_id is 96131 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.683+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b04003520 con 0x7f9b18137900 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 886083898 0 0) 0x7f9b181309f0 con 0x7f9b181383f0 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): my global_id is 76989 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b18132d20 con 0x7f9b181383f0 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4119540532 0 0) 0x7f9b10001110 con 0x7f9b181d2f00 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): my global_id is 6562 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b181309f0 con 0x7f9b181d2f00 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3591770721 0 0) 0x7f9b04003520 con 0x7f9b18137900 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9b10001110 con 0x7f9b18137900 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9b08001df0 con 0x7f9b18137900 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): dump: 2024-05-08T15:55:44.693 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.694 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2808227202 0 0) 0x7f9b18132d20 con 0x7f9b181383f0 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f9b04003520 con 0x7f9b181383f0 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9b0c004b20 con 0x7f9b181383f0 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient(hunting): dump: 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2907345163 0 0) 0x7f9b10001110 con 0x7f9b18137900 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 >> v1:172.21.15.81:6790/0 conn(0x7f9b181383f0 legacy=0x7f9b181302e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 >> v1:172.21.15.81:6789/0 conn(0x7f9b181d2f00 legacy=0x7f9b181d52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 monclient: found mon.b 2024-05-08T15:55:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b180a1b20 con 0x7f9b18137900 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: _finish_auth 0 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b1dd02640 5 monclient: authenticate success, global_id 96131 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b1dd02640 10 monclient: _renew_subs 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b1dd02640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f9b180a1f00 con 0x7f9b18137900 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b1dd02640 10 monclient: _renew_subs 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b1dd02640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f9b18132a40 con 0x7f9b18137900 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f9b08004e10 con 0x7f9b18137900 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:44.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f9b08005300 con 0x7f9b18137900 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9b15ffb640 4 set_mon_vals no callback set 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.684+0000 7f9afffff640 10 monclient: dump: 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.698 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f9b080054a0 con 0x7f9b18137900 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9b1dd02640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f9af8000f80 con 0x7f9b18137900 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f9b08046f30 con 0x7f9b18137900 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6804/3231689356 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b04044820 con 0x7f9b04040bb0 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9b16ffd640 10 monclient: get_auth_request con 0x7f9b04039fd0 auth_method 0 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_get_version_reply(handle=1 version=323) ==== 24+0+0 (unknown 4002016596 0 0) 0x7f9b080158d0 con 0x7f9b18137900 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9afffff640 10 monclient: handle_get_version_reply finishing 1 version 323 2024-05-08T15:55:44.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.685+0000 7f9b17fff640 10 monclient: get_auth_request con 0x7f9b04040bb0 auth_method 0 2024-05-08T15:55:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.686+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== osd.0 v1:172.21.15.7:6804/3231689356 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f9b04044820 con 0x7f9b04040bb0 2024-05-08T15:55:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.699+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 --> v1:172.21.15.7:6804/3231689356 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f9af8002cb0 con 0x7f9b04040bb0 2024-05-08T15:55:44.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9afffff640 1 -- 172.21.15.7:0/3117893446 <== osd.0 v1:172.21.15.7:6804/3231689356 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f9af8002cb0 con 0x7f9b04040bb0 2024-05-08T15:55:44.701 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:44.701 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T15:55:44.701 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:44.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 >> v1:172.21.15.7:6804/3231689356 conn(0x7f9b04040bb0 legacy=0x7f9b04042fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 >> v1:172.21.15.81:6812/847627118 conn(0x7f9b04039fd0 legacy=0x7f9b0403c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 10 monclient: shutdown 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 >> v1:172.21.15.7:6790/0 conn(0x7f9b18137900 legacy=0x7f9b1812fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b17fff640 1 -- 172.21.15.7:0/3117893446 reap_dead start 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 shutdown_connections 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 >> 172.21.15.7:0/3117893446 conn(0x7f9b180a9ec0 msgr2=0x7f9b1812e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 shutdown_connections 2024-05-08T15:55:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.700+0000 7f9b1dd02640 1 -- 172.21.15.7:0/3117893446 wait complete. 2024-05-08T15:55:44.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T15:55:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.816+0000 7f43d65e6640 10 monclient: get_monmap_and_config 2024-05-08T15:55:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.816+0000 7f43d65e6640 10 monclient: build_initial_monmap 2024-05-08T15:55:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.816+0000 7f43d65e6640 10 monclient: monmap: 2024-05-08T15:55:44.812 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:44.817910+0000 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:44.817910+0000 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:44.813 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.817+0000 7f43d65e6640 1 Processor -- start 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.817+0000 7f43d65e6640 1 -- start start 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.817+0000 7f43d65e6640 10 monclient: init 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 10 monclient: _add_conns ranks=[0,7,6,4,1,3,8,5,2] 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 10 monclient(hunting): picked mon.noname-a con 0x7f43d00a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 10 monclient(hunting): picked mon.noname-i con 0x7f43d00a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 10 monclient(hunting): picked mon.noname-d con 0x7f43d013e310 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d00a1660 con 0x7f43d00a53a0 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d00a28f0 con 0x7f43d013e310 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d00a2bd0 con 0x7f43d00a5780 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d65e6640 10 monclient(hunting): authenticate will time out at 1352.933059s 2024-05-08T15:55:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cffff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f43d00a53a0 0x7f43d00a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41654/0 (socket says 172.21.15.7:41654) 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43d4b83640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f43d013e310 0x7f43d0140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:43844/0 (socket says 172.21.15.7:43844) 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cffff640 1 -- 172.21.15.7:0/284842922 learned_addr learned my addr 172.21.15.7:0/284842922 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4111600761 0 0) 0x7f43d00a1660 con 0x7f43d00a53a0 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): my global_id is 95922 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43bc003520 con 0x7f43d00a53a0 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 719551297 0 0) 0x7f43d00a28f0 con 0x7f43d013e310 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): my global_id is 124692 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d00a1660 con 0x7f43d013e310 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4069064521 0 0) 0x7f43d00a2bd0 con 0x7f43d00a5780 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 10 monclient(hunting): my global_id is 115333 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.818+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d00a28f0 con 0x7f43d00a5780 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1798269755 0 0) 0x7f43bc003520 con 0x7f43d00a53a0 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 >> v1:172.21.15.81:6792/0 conn(0x7f43d00a5780 legacy=0x7f43d013cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 >> v1:172.21.15.7:6792/0 conn(0x7f43d013e310 legacy=0x7f43d0140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 monclient: found mon.noname-a 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:44.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d00a6730 con 0x7f43d00a53a0 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: _finish_auth 0 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f43c00017b0 con 0x7f43d00a53a0 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 5 monclient: authenticate success, global_id 95922 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: dump: 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.824 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f43c0001a10 con 0x7f43d00a53a0 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 1 -- 172.21.15.7:0/284842922 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f43c0004be0 con 0x7f43d00a53a0 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 4 set_mon_vals no callback set 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43cf7fe640 10 monclient: dump: 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 10 monclient: shutdown 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 1 -- 172.21.15.7:0/284842922 >> v1:172.21.15.7:6789/0 conn(0x7f43d00a53a0 legacy=0x7f43d00a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 1 -- 172.21.15.7:0/284842922 shutdown_connections 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 1 -- 172.21.15.7:0/284842922 >> 172.21.15.7:0/284842922 conn(0x7f43d00a9fa0 msgr2=0x7f43d00aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 1 -- 172.21.15.7:0/284842922 shutdown_connections 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.819+0000 7f43d65e6640 1 -- 172.21.15.7:0/284842922 wait complete. 2024-05-08T15:55:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.820+0000 7f43d65e6640 10 monclient: build_initial_monmap 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.820+0000 7f43d65e6640 10 monclient: monmap: 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.820+0000 7f43d65e6640 1 Processor -- start 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.820+0000 7f43d65e6640 1 -- start start 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.820+0000 7f43d65e6640 10 monclient: init 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 10 monclient: _add_conns ranks=[3,0,1,4,2,7,5,8,6] 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 10 monclient(hunting): picked mon.noname-g con 0x7f43d00a53a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 10 monclient(hunting): picked mon.noname-a con 0x7f43d00a5780 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 10 monclient(hunting): picked mon.noname-f con 0x7f43d013e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d0072200 con 0x7f43d00a5780 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d01d1b70 con 0x7f43d013e310 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d0070690 con 0x7f43d00a53a0 2024-05-08T15:55:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43d65e6640 10 monclient(hunting): authenticate will time out at 1352.935839s 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43c7fff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f43d00a5780 0x7f43d01d1460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41668/0 (socket says 172.21.15.7:41668) 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43c7fff640 1 -- 172.21.15.7:0/2211726772 learned_addr learned my addr 172.21.15.7:0/2211726772 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2542708671 0 0) 0x7f43d0072200 con 0x7f43d00a5780 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): my global_id is 95931 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43b0003520 con 0x7f43d00a5780 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1305814536 0 0) 0x7f43d0070690 con 0x7f43d00a53a0 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): my global_id is 76998 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d0072200 con 0x7f43d00a53a0 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066943306 0 0) 0x7f43d01d1b70 con 0x7f43d013e310 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _negotiate 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): my global_id is 6571 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43d0070690 con 0x7f43d013e310 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 857032575 0 0) 0x7f43b0003520 con 0x7f43d00a5780 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43d01d1b70 con 0x7f43d00a5780 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f43b8001b30 con 0x7f43d00a5780 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): dump: 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.831 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.832 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 710309009 0 0) 0x7f43d0070690 con 0x7f43d013e310 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43b0003520 con 0x7f43d013e310 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3452708976 0 0) 0x7f43d0072200 con 0x7f43d00a53a0 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f43d0070690 con 0x7f43d00a53a0 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f43c8001cf0 con 0x7f43d013e310 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.821+0000 7f43cd7fa640 10 monclient(hunting): dump: 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.833 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f43c0004b20 con 0x7f43d00a53a0 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:44.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient(hunting): dump: 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.835 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3579456860 0 0) 0x7f43d01d1b70 con 0x7f43d00a5780 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 >> v1:172.21.15.81:6790/0 conn(0x7f43d00a53a0 legacy=0x7f43d01d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 >> v1:172.21.15.81:6789/0 conn(0x7f43d013e310 legacy=0x7f43d01d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: _finish_hunting 0 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 monclient: found mon.a 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d00a1e00 con 0x7f43d00a5780 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: _finish_auth 0 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: _check_auth_tickets 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43d65e6640 5 monclient: authenticate success, global_id 95931 2024-05-08T15:55:44.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43d65e6640 10 monclient: _renew_subs 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43d65e6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f43d00a43c0 con 0x7f43d00a5780 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43d65e6640 10 monclient: _renew_subs 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43d65e6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f43d00708d0 con 0x7f43d00a5780 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f43b80038c0 con 0x7f43d00a5780 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f43b8003db0 con 0x7f43d00a5780 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43ceffd640 4 set_mon_vals no callback set 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.822+0000 7f43cd7fa640 10 monclient: dump: 2024-05-08T15:55:44.837 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:44.838 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.823+0000 7f43d65e6640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.823+0000 7f43d65e6640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.823+0000 7f43d65e6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.823+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f43b8003f50 con 0x7f43d00a5780 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.823+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b4005080 con 0x7f43d00a5780 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.823+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f43b8046d90 con 0x7f43d00a5780 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.823+0000 7f43cffff640 10 monclient: get_auth_request con 0x7f43b003a4b0 auth_method 0 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.827+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f43b8015f80 con 0x7f43d00a5780 2024-05-08T15:55:44.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.827+0000 7f43cd7fa640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:44.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:44.827+0000 7f43cd7fa640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:45.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.011+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f43b4000b90 con 0x7f43b003a4b0 2024-05-08T15:55:45.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.011+0000 7f43cd7fa640 1 -- 172.21.15.7:0/2211726772 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (unknown 2057319476 0 0) 0x7f43b4000b90 con 0x7f43b003a4b0 2024-05-08T15:55:45.007 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T15:55:45.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 >> v1:172.21.15.81:6812/847627118 conn(0x7f43b003a4b0 legacy=0x7f43b003c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 10 monclient: shutdown 2024-05-08T15:55:45.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 >> v1:172.21.15.7:6789/0 conn(0x7f43d00a5780 legacy=0x7f43d01d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 shutdown_connections 2024-05-08T15:55:45.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 >> 172.21.15.7:0/2211726772 conn(0x7f43d00a9fa0 msgr2=0x7f43d00a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 shutdown_connections 2024-05-08T15:55:45.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.013+0000 7f43d65e6640 1 -- 172.21.15.7:0/2211726772 wait complete. 2024-05-08T15:55:45.020 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T15:55:45.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T15:55:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.130+0000 7fd1686c8640 10 monclient: get_monmap_and_config 2024-05-08T15:55:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.130+0000 7fd1686c8640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.130+0000 7fd1686c8640 10 monclient: monmap: 2024-05-08T15:55:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:45.131549+0000 2024-05-08T15:55:45.126 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:45.131549+0000 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.127 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 1 Processor -- start 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 1 -- start start 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient: init 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient: _add_conns ranks=[6,7,3,2,5,1,4,0,8] 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient(hunting): picked mon.noname-d con 0x7fd1600a5250 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient(hunting): picked mon.noname-i con 0x7fd1600a48c0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient(hunting): picked mon.noname-g con 0x7fd16013e340 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1600a28a0 con 0x7fd16013e340 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1600a2b80 con 0x7fd1600a5250 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1600a2e60 con 0x7fd1600a48c0 2024-05-08T15:55:45.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd1686c8640 10 monclient(hunting): authenticate will time out at 1353.246507s 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.131+0000 7fd166464640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fd1600a5250 0x7fd1600a5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:43856/0 (socket says 172.21.15.7:43856) 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd166464640 1 -- 172.21.15.7:0/4093142066 learned_addr learned my addr 172.21.15.7:0/4093142066 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907018891 0 0) 0x7fd1600a2b80 con 0x7fd1600a5250 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient(hunting): my global_id is 124701 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd14c003520 con 0x7fd1600a5250 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3678872813 0 0) 0x7fd14c003520 con 0x7fd1600a5250 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 >> v1:172.21.15.81:6792/0 conn(0x7fd1600a48c0 legacy=0x7fd16013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 legacy=0x7fd160140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd166c65640 1 -- 172.21.15.7:0/4093142066 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 legacy=0x7fd160140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 15 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd166c65640 1 -- 172.21.15.7:0/4093142066 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 legacy=0x7fd160140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd166c65640 1 --1- 172.21.15.7:0/4093142066 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 0x7fd160140730 :-1 s=OPENED pgs=553 cs=1 l=1).handle_message read tag failed 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd166c65640 1 --1- 172.21.15.7:0/4093142066 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 0x7fd160140730 :-1 s=OPENED pgs=553 cs=1 l=1).fault on lossy channel, failing 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 monclient: found mon.noname-d 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1600a42c0 con 0x7fd1600a5250 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: ms_handle_reset stray mon v1:172.21.15.81:6790/0 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd15c0017b0 con 0x7fd1600a5250 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd1686c8640 5 monclient: authenticate success, global_id 124701 2024-05-08T15:55:45.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd1686c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: dump: 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.138 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd1686c8640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd15c001aa0 con 0x7fd1600a5250 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 1 -- 172.21.15.7:0/4093142066 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd15c004be0 con 0x7fd1600a5250 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd1686c8640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd1686c8640 4 set_mon_vals no callback set 2024-05-08T15:55:45.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.132+0000 7fd165462640 10 monclient: dump: 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.140 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 10 monclient: shutdown 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 1 -- 172.21.15.7:0/4093142066 >> v1:172.21.15.7:6792/0 conn(0x7fd1600a5250 legacy=0x7fd1600a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 1 -- 172.21.15.7:0/4093142066 shutdown_connections 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 1 -- 172.21.15.7:0/4093142066 >> 172.21.15.7:0/4093142066 conn(0x7fd1600a9ec0 msgr2=0x7fd1600aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 1 -- 172.21.15.7:0/4093142066 shutdown_connections 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 1 -- 172.21.15.7:0/4093142066 wait complete. 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.133+0000 7fd1686c8640 10 monclient: monmap: 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:45.141 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.142 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 1 Processor -- start 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 1 -- start start 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient: init 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient: _add_conns ranks=[5,1,3,0,2,7,4,6,8] 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient(hunting): picked mon.noname-h con 0x7fd1600a48c0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient(hunting): picked mon.noname-f con 0x7fd1600a5250 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient(hunting): picked mon.noname-g con 0x7fd16013e340 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd160133d20 con 0x7fd1600a5250 2024-05-08T15:55:45.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd160134660 con 0x7fd16013e340 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1600a1b20 con 0x7fd1600a48c0 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd1686c8640 10 monclient(hunting): authenticate will time out at 1353.249309s 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd165c63640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd1600a5250 0x7fd160133610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:34062/0 (socket says 172.21.15.7:34062) 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd166c65640 1 --1- >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 0x7fd1601d4720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6790/0 says I am v1:172.21.15.7:44486/0 (socket says 172.21.15.7:44486) 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.134+0000 7fd165c63640 1 -- 172.21.15.7:0/2750361141 learned_addr learned my addr 172.21.15.7:0/2750361141 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd166c65640 1 -- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 legacy=0x7fd1601d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd166c65640 1 -- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 legacy=0x7fd1601d4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd166c65640 1 --1- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 0x7fd1601d4720 :-1 s=OPENED pgs=554 cs=1 l=1).handle_message read tag failed 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd166c65640 1 --1- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 0x7fd1601d4720 :-1 s=OPENED pgs=554 cs=1 l=1).fault on lossy channel, failing 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127295699 0 0) 0x7fd160133d20 con 0x7fd1600a5250 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): my global_id is 6580 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd144003520 con 0x7fd1600a5250 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): ms_handle_reset hunted mon v1:172.21.15.81:6790/0 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4276813352 0 0) 0x7fd1600a1b20 con 0x7fd1600a48c0 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): my global_id is 6746 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd160133d20 con 0x7fd1600a48c0 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2286004225 0 0) 0x7fd144003520 con 0x7fd1600a5250 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd1600a1b20 con 0x7fd1600a5250 2024-05-08T15:55:45.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd150001ab0 con 0x7fd1600a5250 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.146 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1241542991 0 0) 0x7fd160133d20 con 0x7fd1600a48c0 2024-05-08T15:55:45.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd144003520 con 0x7fd1600a48c0 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd15c001770 con 0x7fd1600a48c0 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.148 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3757701301 0 0) 0x7fd1600a1b20 con 0x7fd1600a5250 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6791/0 conn(0x7fd1600a48c0 legacy=0x7fd160132f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6790/0 conn(0x7fd16013e340 legacy=0x7fd1601d4720 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 monclient: found mon.f 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1600a1ee0 con 0x7fd1600a5250 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd1686c8640 5 monclient: authenticate success, global_id 6580 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd1686c8640 10 monclient: _renew_subs 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd1686c8640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd1600a03d0 con 0x7fd1600a5250 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd1686c8640 10 monclient: _renew_subs 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd1686c8640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fd160135b10 con 0x7fd1600a5250 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd1500038c0 con 0x7fd1600a5250 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd150003e30 con 0x7fd1600a5250 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd164c61640 4 set_mon_vals no callback set 2024-05-08T15:55:45.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.135+0000 7fd156ffd640 10 monclient: dump: 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.151 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.152 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.136+0000 7fd1686c8640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.136+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.81:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fd148000f80 con 0x7fd1600a5250 2024-05-08T15:55:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.136+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd150003fd0 con 0x7fd1600a5250 2024-05-08T15:55:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.136+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fd150046cb0 con 0x7fd1600a5250 2024-05-08T15:55:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.137+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.7:6800/601836183 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd144044960 con 0x7fd144040cf0 2024-05-08T15:55:45.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.137+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_get_version_reply(handle=1 version=323) ==== 24+0+0 (unknown 4002016596 0 0) 0x7fd150047030 con 0x7fd1600a5250 2024-05-08T15:55:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.137+0000 7fd156ffd640 10 monclient: handle_get_version_reply finishing 1 version 323 2024-05-08T15:55:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.137+0000 7fd166464640 10 monclient: get_auth_request con 0x7fd14403a110 auth_method 0 2024-05-08T15:55:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.137+0000 7fd166c65640 10 monclient: get_auth_request con 0x7fd144040cf0 auth_method 0 2024-05-08T15:55:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.137+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== osd.1 v1:172.21.15.7:6800/601836183 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fd144044960 con 0x7fd144040cf0 2024-05-08T15:55:45.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 --> v1:172.21.15.7:6800/601836183 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd148002cb0 con 0x7fd144040cf0 2024-05-08T15:55:45.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd156ffd640 1 -- 172.21.15.7:0/2750361141 <== osd.1 v1:172.21.15.7:6800/601836183 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fd148002cb0 con 0x7fd144040cf0 2024-05-08T15:55:45.154 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:45.154 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T15:55:45.154 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 >> v1:172.21.15.7:6800/601836183 conn(0x7fd144040cf0 legacy=0x7fd144043110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6812/847627118 conn(0x7fd14403a110 legacy=0x7fd14403c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 10 monclient: shutdown 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 >> v1:172.21.15.81:6789/0 conn(0x7fd1600a5250 legacy=0x7fd160133610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd166c65640 1 -- 172.21.15.7:0/2750361141 reap_dead start 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 shutdown_connections 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 >> 172.21.15.7:0/2750361141 conn(0x7fd1600a9ec0 msgr2=0x7fd1600a7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 shutdown_connections 2024-05-08T15:55:45.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.151+0000 7fd1686c8640 1 -- 172.21.15.7:0/2750361141 wait complete. 2024-05-08T15:55:45.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T15:55:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.265+0000 7f7a49921640 10 monclient: get_monmap_and_config 2024-05-08T15:55:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.265+0000 7f7a49921640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.265+0000 7f7a49921640 10 monclient: monmap: 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:45.266949+0000 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:45.266949+0000 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.263 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.266+0000 7f7a49921640 1 Processor -- start 2024-05-08T15:55:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.266+0000 7f7a49921640 1 -- start start 2024-05-08T15:55:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.266+0000 7f7a49921640 10 monclient: init 2024-05-08T15:55:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 10 monclient: _add_conns ranks=[1,3,2,8,5,7,6,0,4] 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 10 monclient(hunting): picked mon.noname-f con 0x7f7a440a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 10 monclient(hunting): picked mon.noname-g con 0x7f7a440a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 10 monclient(hunting): picked mon.noname-b con 0x7f7a4413e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a440a1660 con 0x7f7a440a53a0 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a440a28f0 con 0x7f7a4413e310 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a440a2bd0 con 0x7f7a440a5780 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 10 monclient(hunting): authenticate will time out at 1353.381888s 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a437fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f7a4413e310 0x7f7a44140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37040/0 (socket says 172.21.15.7:37040) 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a437fe640 1 -- 172.21.15.7:0/815388835 learned_addr learned my addr 172.21.15.7:0/815388835 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3367466184 0 0) 0x7f7a440a28f0 con 0x7f7a4413e310 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): my global_id is 96140 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a2c003520 con 0x7f7a4413e310 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4013745098 0 0) 0x7f7a440a1660 con 0x7f7a440a53a0 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): my global_id is 6589 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a440a28f0 con 0x7f7a440a53a0 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3260584123 0 0) 0x7f7a440a2bd0 con 0x7f7a440a5780 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient(hunting): my global_id is 77025 2024-05-08T15:55:45.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a440a1660 con 0x7f7a440a5780 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4071810898 0 0) 0x7f7a2c003520 con 0x7f7a4413e310 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 >> v1:172.21.15.81:6790/0 conn(0x7f7a440a5780 legacy=0x7f7a4413cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 >> v1:172.21.15.81:6789/0 conn(0x7f7a440a53a0 legacy=0x7f7a440a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 monclient: found mon.noname-b 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a440a6730 con 0x7f7a4413e310 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 5 monclient: authenticate success, global_id 96140 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a3c001ab0 con 0x7f7a4413e310 2024-05-08T15:55:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a41ffb640 10 monclient: dump: 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.267+0000 7f7a49921640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7a3c001d10 con 0x7f7a4413e310 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a41ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a41ffb640 1 -- 172.21.15.7:0/815388835 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a3c004f60 con 0x7f7a4413e310 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 4 set_mon_vals no callback set 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a41ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a41ffb640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a41ffb640 10 monclient: dump: 2024-05-08T15:55:45.275 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 10 monclient: shutdown 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 1 -- 172.21.15.7:0/815388835 >> v1:172.21.15.7:6790/0 conn(0x7f7a4413e310 legacy=0x7f7a44140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 1 -- 172.21.15.7:0/815388835 shutdown_connections 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 1 -- 172.21.15.7:0/815388835 >> 172.21.15.7:0/815388835 conn(0x7f7a440a9fa0 msgr2=0x7f7a440aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 1 -- 172.21.15.7:0/815388835 shutdown_connections 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.268+0000 7f7a49921640 1 -- 172.21.15.7:0/815388835 wait complete. 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.269+0000 7f7a49921640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.269+0000 7f7a49921640 10 monclient: monmap: 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:45.277 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.269+0000 7f7a49921640 1 Processor -- start 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.269+0000 7f7a49921640 1 -- start start 2024-05-08T15:55:45.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.269+0000 7f7a49921640 10 monclient: init 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 10 monclient: _add_conns ranks=[6,3,0,5,8,1,2,7,4] 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 10 monclient(hunting): picked mon.noname-d con 0x7f7a440a53a0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 10 monclient(hunting): picked mon.noname-g con 0x7f7a440a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 10 monclient(hunting): picked mon.noname-a con 0x7f7a4413e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a44135ba0 con 0x7f7a4413e310 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a44134ee0 con 0x7f7a440a5780 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a440a1e00 con 0x7f7a440a53a0 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a49921640 10 monclient(hunting): authenticate will time out at 1353.384777s 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a42ffd640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7f7a440a53a0 0x7f7a4412e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:43868/0 (socket says 172.21.15.7:43868) 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a42ffd640 1 -- 172.21.15.7:0/1875176597 learned_addr learned my addr 172.21.15.7:0/1875176597 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a437fe640 1 --1- 172.21.15.7:0/1875176597 >> v1:172.21.15.7:6789/0 conn(0x7f7a4413e310 0x7f7a441d5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41684/0 (socket says 172.21.15.7:41684) 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4088032988 0 0) 0x7f7a440a1e00 con 0x7f7a440a53a0 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): my global_id is 124710 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a30003520 con 0x7f7a440a53a0 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139404522 0 0) 0x7f7a440a6730 con 0x7f7a4413e310 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): my global_id is 95940 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a440a1e00 con 0x7f7a4413e310 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2208855413 0 0) 0x7f7a44134ee0 con 0x7f7a440a5780 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): my global_id is 77034 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a440a6730 con 0x7f7a440a5780 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 533674647 0 0) 0x7f7a30003520 con 0x7f7a440a53a0 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a44134ee0 con 0x7f7a440a53a0 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a34001b10 con 0x7f7a440a53a0 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): dump: 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.281 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3633263882 0 0) 0x7f7a440a1e00 con 0x7f7a4413e310 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a30003520 con 0x7f7a4413e310 2024-05-08T15:55:45.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a3c001a30 con 0x7f7a4413e310 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 10 monclient(hunting): dump: 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.283 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.270+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2428699565 0 0) 0x7f7a440a6730 con 0x7f7a440a5780 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a440a1e00 con 0x7f7a440a5780 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a38001900 con 0x7f7a440a5780 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient(hunting): dump: 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.284 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.285 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3876414410 0 0) 0x7f7a44134ee0 con 0x7f7a440a53a0 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 >> v1:172.21.15.81:6790/0 conn(0x7f7a440a5780 legacy=0x7f7a441d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 >> v1:172.21.15.7:6789/0 conn(0x7f7a4413e310 legacy=0x7f7a441d5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 monclient: found mon.d 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a440a43c0 con 0x7f7a440a53a0 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a49921640 5 monclient: authenticate success, global_id 124710 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a49921640 10 monclient: _renew_subs 2024-05-08T15:55:45.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a49921640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6792/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7a44134920 con 0x7f7a440a53a0 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a49921640 10 monclient: _renew_subs 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a49921640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6792/0 -- mon_subscribe({osdmap=0}) -- 0x7f7a44133a30 con 0x7f7a440a53a0 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7a340038c0 con 0x7f7a440a53a0 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a34003e30 con 0x7f7a440a53a0 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 10 monclient: dump: 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.287 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a417fa640 4 set_mon_vals no callback set 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.271+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7a34003fd0 con 0x7f7a440a53a0 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.272+0000 7f7a49921640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:45.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.272+0000 7f7a49921640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.272+0000 7f7a49921640 10 monclient: _send_mon_message to mon.d at v1:172.21.15.7:6792/0 2024-05-08T15:55:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.272+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.7:6792/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a24005080 con 0x7f7a440a53a0 2024-05-08T15:55:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.272+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f7a34046c40 con 0x7f7a440a53a0 2024-05-08T15:55:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.272+0000 7f7a427fc640 10 monclient: get_auth_request con 0x7f7a3003a470 auth_method 0 2024-05-08T15:55:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.276+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mon.6 v1:172.21.15.7:6792/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f7a34015f10 con 0x7f7a440a53a0 2024-05-08T15:55:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.276+0000 7f7a2bfff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.276+0000 7f7a2bfff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:45.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.461+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f7a24000b90 con 0x7f7a3003a470 2024-05-08T15:55:45.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.461+0000 7f7a2bfff640 1 -- 172.21.15.7:0/1875176597 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (unknown 489605429 0 0) 0x7f7a24000b90 con 0x7f7a3003a470 2024-05-08T15:55:45.458 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 >> v1:172.21.15.81:6812/847627118 conn(0x7f7a3003a470 legacy=0x7f7a3003c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 10 monclient: shutdown 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 >> v1:172.21.15.7:6792/0 conn(0x7f7a440a53a0 legacy=0x7f7a4412e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 shutdown_connections 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 >> 172.21.15.7:0/1875176597 conn(0x7f7a440a9fa0 msgr2=0x7f7a440a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 shutdown_connections 2024-05-08T15:55:45.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.464+0000 7f7a49921640 1 -- 172.21.15.7:0/1875176597 wait complete. 2024-05-08T15:55:45.470 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T15:55:45.470 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T15:55:45.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.582+0000 7fa104559640 10 monclient: get_monmap_and_config 2024-05-08T15:55:45.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.582+0000 7fa104559640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.582+0000 7fa104559640 10 monclient: monmap: 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:45.583092+0000 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:45.583092+0000 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.578 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.579 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.579 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.579 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.582+0000 7fa104559640 1 Processor -- start 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.582+0000 7fa104559640 1 -- start start 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.582+0000 7fa104559640 10 monclient: init 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 10 monclient: _add_conns ranks=[8,0,2,5,6,3,4,1,7] 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 10 monclient(hunting): picked mon.noname-e con 0x7fa0fc137900 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 10 monclient(hunting): picked mon.noname-a con 0x7fa0fc0a4110 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 10 monclient(hunting): picked mon.noname-b con 0x7fa0fc1383f0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc0a48c0 con 0x7fa0fc0a4110 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc0a28a0 con 0x7fa0fc1383f0 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc0a2b80 con 0x7fa0fc137900 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa104559640 10 monclient(hunting): authenticate will time out at 1353.698029s 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa101af4640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa0fc0a4110 0x7fa0fc13ce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41690/0 (socket says 172.21.15.7:41690) 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa101af4640 1 -- 172.21.15.7:0/1929640195 learned_addr learned my addr 172.21.15.7:0/1929640195 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3102119172 0 0) 0x7fa0fc0a28a0 con 0x7fa0fc1383f0 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): my global_id is 96149 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e8003520 con 0x7fa0fc1383f0 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2993049250 0 0) 0x7fa0fc0a48c0 con 0x7fa0fc0a4110 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): my global_id is 95949 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0fc0a28a0 con 0x7fa0fc0a4110 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600582584 0 0) 0x7fa0fc0a2b80 con 0x7fa0fc137900 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 10 monclient(hunting): my global_id is 144233 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.583+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0fc0a48c0 con 0x7fa0fc137900 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2850405783 0 0) 0x7fa0e8003520 con 0x7fa0fc1383f0 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 >> v1:172.21.15.7:6793/0 conn(0x7fa0fc137900 legacy=0x7fa0fc137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 >> v1:172.21.15.7:6789/0 conn(0x7fa0fc0a4110 legacy=0x7fa0fc13ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 monclient: found mon.noname-b 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc0a2e60 con 0x7fa0fc1383f0 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa0f4001b30 con 0x7fa0fc1383f0 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 5 monclient: authenticate success, global_id 96149 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: dump: 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.590 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa0f4001de0 con 0x7fa0fc1383f0 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 1 -- 172.21.15.7:0/1929640195 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa0f4003cd0 con 0x7fa0fc1383f0 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 4 set_mon_vals no callback set 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa1012f3640 10 monclient: dump: 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.592 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 10 monclient: shutdown 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 1 -- 172.21.15.7:0/1929640195 >> v1:172.21.15.7:6790/0 conn(0x7fa0fc1383f0 legacy=0x7fa0fc140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 1 -- 172.21.15.7:0/1929640195 shutdown_connections 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 1 -- 172.21.15.7:0/1929640195 >> 172.21.15.7:0/1929640195 conn(0x7fa0fc0a9ec0 msgr2=0x7fa0fc0aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 1 -- 172.21.15.7:0/1929640195 shutdown_connections 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.584+0000 7fa104559640 1 -- 172.21.15.7:0/1929640195 wait complete. 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.585+0000 7fa104559640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.585+0000 7fa104559640 10 monclient: monmap: 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:45.593 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.594 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.585+0000 7fa104559640 1 Processor -- start 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.585+0000 7fa104559640 1 -- start start 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.585+0000 7fa104559640 10 monclient: init 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 10 monclient: _add_conns ranks=[2,4,6,7,8,3,0,5,1] 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 10 monclient(hunting): picked mon.noname-b con 0x7fa0fc137900 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 10 monclient(hunting): picked mon.noname-c con 0x7fa0fc1383f0 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 10 monclient(hunting): picked mon.noname-d con 0x7fa0fc136610 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc0a1f00 con 0x7fa0fc137900 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc0a1b20 con 0x7fa0fc1383f0 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc121c50 con 0x7fa0fc136610 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa104559640 10 monclient(hunting): authenticate will time out at 1353.700877s 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa1022f5640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fa0fc137900 0x7fa0fc135ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37066/0 (socket says 172.21.15.7:37066) 2024-05-08T15:55:45.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa1022f5640 1 -- 172.21.15.7:0/3618482545 learned_addr learned my addr 172.21.15.7:0/3618482545 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 28277816 0 0) 0x7fa0fc0a1f00 con 0x7fa0fc137900 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): my global_id is 96158 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e0003520 con 0x7fa0fc137900 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 708407307 0 0) 0x7fa0fc0a1b20 con 0x7fa0fc1383f0 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): my global_id is 124519 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0fc0a1f00 con 0x7fa0fc1383f0 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252157275 0 0) 0x7fa0fc121c50 con 0x7fa0fc136610 2024-05-08T15:55:45.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): my global_id is 124719 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0fc0a1b20 con 0x7fa0fc136610 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3343981352 0 0) 0x7fa0e0003520 con 0x7fa0fc137900 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa0fc121c50 con 0x7fa0fc137900 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa0f8001670 con 0x7fa0fc137900 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.597 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2960565977 0 0) 0x7fa0fc0a1f00 con 0x7fa0fc1383f0 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa0e0003520 con 0x7fa0fc1383f0 2024-05-08T15:55:45.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa0ec001510 con 0x7fa0fc1383f0 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.599 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.586+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2013019398 0 0) 0x7fa0fc0a1b20 con 0x7fa0fc136610 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa0fc0a1f00 con 0x7fa0fc136610 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa0f4001400 con 0x7fa0fc136610 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient(hunting): got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient(hunting): dump: 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.600 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4087656167 0 0) 0x7fa0fc121c50 con 0x7fa0fc137900 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 >> v1:172.21.15.7:6792/0 conn(0x7fa0fc136610 legacy=0x7fa0fc1d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 >> v1:172.21.15.7:6791/0 conn(0x7fa0fc1383f0 legacy=0x7fa0fc1d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 monclient: found mon.b 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc122dc0 con 0x7fa0fc137900 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 5 monclient: authenticate success, global_id 96158 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 10 monclient: _renew_subs 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc1363b0 con 0x7fa0fc137900 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 10 monclient: _renew_subs 2024-05-08T15:55:45.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa0f80038a0 con 0x7fa0fc137900 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7fa0fc09fc00 con 0x7fa0fc137900 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa0f8003e10 con 0x7fa0fc137900 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa0f2ffd640 10 monclient: dump: 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.603 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa100af2640 4 set_mon_vals no callback set 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.587+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6790/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fa0fc075820 con 0x7fa0fc137900 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.588+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa0f8003fb0 con 0x7fa0fc137900 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.588+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fa0f8046cc0 con 0x7fa0fc137900 2024-05-08T15:55:45.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.588+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6808/2598303896 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0f4001400 con 0x7fa0e00412c0 2024-05-08T15:55:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.588+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_get_version_reply(handle=1 version=323) ==== 24+0+0 (unknown 4002016596 0 0) 0x7fa0f8047130 con 0x7fa0fc137900 2024-05-08T15:55:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.588+0000 7fa0f2ffd640 10 monclient: handle_get_version_reply finishing 1 version 323 2024-05-08T15:55:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.588+0000 7fa102af6640 10 monclient: get_auth_request con 0x7fa0e00412c0 auth_method 0 2024-05-08T15:55:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.588+0000 7fa101af4640 10 monclient: get_auth_request con 0x7fa0e003a690 auth_method 0 2024-05-08T15:55:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.589+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== osd.2 v1:172.21.15.7:6808/2598303896 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fa0f4001400 con 0x7fa0e00412c0 2024-05-08T15:55:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.602+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 --> v1:172.21.15.7:6808/2598303896 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa0fc09fe90 con 0x7fa0e00412c0 2024-05-08T15:55:45.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.602+0000 7fa0f2ffd640 1 -- 172.21.15.7:0/3618482545 <== osd.2 v1:172.21.15.7:6808/2598303896 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fa0fc09fe90 con 0x7fa0e00412c0 2024-05-08T15:55:45.606 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:45.606 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T15:55:45.606 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.602+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 >> v1:172.21.15.7:6808/2598303896 conn(0x7fa0e00412c0 legacy=0x7fa0e00436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 >> v1:172.21.15.81:6812/847627118 conn(0x7fa0e003a690 legacy=0x7fa0e003cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 10 monclient: shutdown 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 >> v1:172.21.15.7:6790/0 conn(0x7fa0fc137900 legacy=0x7fa0fc135ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa102af6640 1 -- 172.21.15.7:0/3618482545 reap_dead start 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 shutdown_connections 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 >> 172.21.15.7:0/3618482545 conn(0x7fa0fc0a9ec0 msgr2=0x7fa0fc0a96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 shutdown_connections 2024-05-08T15:55:45.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.603+0000 7fa104559640 1 -- 172.21.15.7:0/3618482545 wait complete. 2024-05-08T15:55:45.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T15:55:45.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.717+0000 7fd79d725640 10 monclient: get_monmap_and_config 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.717+0000 7fd79d725640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.717+0000 7fd79d725640 10 monclient: monmap: 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:45.718123+0000 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:45.718123+0000 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.714 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.717+0000 7fd79d725640 1 Processor -- start 2024-05-08T15:55:45.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.717+0000 7fd79d725640 1 -- start start 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.717+0000 7fd79d725640 10 monclient: init 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 10 monclient: _add_conns ranks=[0,1,4,8,7,2,3,6,5] 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 10 monclient(hunting): picked mon.noname-a con 0x7fd7980a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 10 monclient(hunting): picked mon.noname-f con 0x7fd7980a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 10 monclient(hunting): picked mon.noname-c con 0x7fd79813e310 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7980a1660 con 0x7fd7980a53a0 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7980a28f0 con 0x7fd7980a5780 2024-05-08T15:55:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7980a2bd0 con 0x7fd79813e310 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 10 monclient(hunting): authenticate will time out at 1353.832934s 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd7977fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fd79813e310 0x7fd798140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:44986/0 (socket says 172.21.15.7:44986) 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd7977fe640 1 -- 172.21.15.7:0/470414841 learned_addr learned my addr 172.21.15.7:0/470414841 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd796ffd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fd7980a53a0 0x7fd7980a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41706/0 (socket says 172.21.15.7:41706) 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4067088496 0 0) 0x7fd7980a1660 con 0x7fd7980a53a0 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): my global_id is 95958 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd780003520 con 0x7fd7980a53a0 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2999324581 0 0) 0x7fd7980a2bd0 con 0x7fd79813e310 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): my global_id is 124528 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7980a1660 con 0x7fd79813e310 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 35185688 0 0) 0x7fd7980a28f0 con 0x7fd7980a5780 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient(hunting): my global_id is 6598 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7980a2bd0 con 0x7fd7980a5780 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4160738584 0 0) 0x7fd780003520 con 0x7fd7980a53a0 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 >> v1:172.21.15.7:6791/0 conn(0x7fd79813e310 legacy=0x7fd798140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 >> v1:172.21.15.81:6789/0 conn(0x7fd7980a5780 legacy=0x7fd79813cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 monclient: found mon.noname-a 2024-05-08T15:55:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7980a6730 con 0x7fd7980a53a0 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd7880016e0 con 0x7fd7980a53a0 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 5 monclient: authenticate success, global_id 95958 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd79d725640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.718+0000 7fd795ffb640 10 monclient: dump: 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd788001940 con 0x7fd7980a53a0 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd795ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd795ffb640 1 -- 172.21.15.7:0/470414841 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd7880047e0 con 0x7fd7980a53a0 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 4 set_mon_vals no callback set 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd795ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd795ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd795ffb640 10 monclient: dump: 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.728 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 10 monclient: shutdown 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 1 -- 172.21.15.7:0/470414841 >> v1:172.21.15.7:6789/0 conn(0x7fd7980a53a0 legacy=0x7fd7980a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 1 -- 172.21.15.7:0/470414841 shutdown_connections 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 1 -- 172.21.15.7:0/470414841 >> 172.21.15.7:0/470414841 conn(0x7fd7980a9fa0 msgr2=0x7fd7980aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 1 -- 172.21.15.7:0/470414841 shutdown_connections 2024-05-08T15:55:45.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.719+0000 7fd79d725640 1 -- 172.21.15.7:0/470414841 wait complete. 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.720+0000 7fd79d725640 10 monclient: build_initial_monmap 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.720+0000 7fd79d725640 10 monclient: monmap: 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:45.730 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.720+0000 7fd79d725640 1 Processor -- start 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.720+0000 7fd79d725640 1 -- start start 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.720+0000 7fd79d725640 10 monclient: init 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 10 monclient: _add_conns ranks=[7,8,1,2,0,5,4,3,6] 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 10 monclient(hunting): picked mon.noname-i con 0x7fd7980a53a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 10 monclient(hunting): picked mon.noname-e con 0x7fd7980a5780 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 10 monclient(hunting): picked mon.noname-f con 0x7fd79813e310 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd798138640 con 0x7fd79813e310 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd79813ca40 con 0x7fd7980a53a0 2024-05-08T15:55:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd798132ce0 con 0x7fd7980a5780 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd79d725640 10 monclient(hunting): authenticate will time out at 1353.835973s 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd7967fc640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fd7980a5780 0x7fd79813c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55794/0 (socket says 172.21.15.7:55794) 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd7967fc640 1 -- 172.21.15.7:0/4186696458 learned_addr learned my addr 172.21.15.7:0/4186696458 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375553274 0 0) 0x7fd798132ce0 con 0x7fd7980a5780 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): my global_id is 144242 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd784003520 con 0x7fd7980a5780 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2812211007 0 0) 0x7fd798138640 con 0x7fd79813e310 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): my global_id is 6607 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd798132ce0 con 0x7fd79813e310 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 806005933 0 0) 0x7fd79813ca40 con 0x7fd7980a53a0 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): my global_id is 115351 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd798138640 con 0x7fd7980a53a0 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2644472483 0 0) 0x7fd784003520 con 0x7fd7980a5780 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd79813ca40 con 0x7fd7980a5780 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd78c001b30 con 0x7fd7980a5780 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): dump: 2024-05-08T15:55:45.733 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.734 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.721+0000 7fd77f7fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 867779590 0 0) 0x7fd79813ca40 con 0x7fd7980a5780 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 >> v1:172.21.15.81:6792/0 conn(0x7fd7980a53a0 legacy=0x7fd79813bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 >> v1:172.21.15.81:6789/0 conn(0x7fd79813e310 legacy=0x7fd7981d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: _finish_hunting 0 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 monclient: found mon.e 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7981343c0 con 0x7fd7980a5780 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: _finish_auth 0 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: _check_auth_tickets 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd79d725640 5 monclient: authenticate success, global_id 144242 2024-05-08T15:55:45.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd79d725640 10 monclient: _renew_subs 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd79d725640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd798134980 con 0x7fd7980a5780 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd79d725640 10 monclient: _renew_subs 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd79d725640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fd79813d6b0 con 0x7fd7980a5780 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd78c0038c0 con 0x7fd7980a5780 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd78c003e30 con 0x7fd7980a5780 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd7957fa640 4 set_mon_vals no callback set 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 10 monclient: dump: 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:45.736 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:45.737 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.722+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd78c004050 con 0x7fd7980a5780 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.723+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fd78c046de0 con 0x7fd7980a5780 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.723+0000 7fd79d725640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.723+0000 7fd79d725640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.723+0000 7fd79d725640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.723+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd778005080 con 0x7fd7980a5780 2024-05-08T15:55:45.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.723+0000 7fd796ffd640 10 monclient: get_auth_request con 0x7fd7840399c0 auth_method 0 2024-05-08T15:55:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.727+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fd78c015ff0 con 0x7fd7980a5780 2024-05-08T15:55:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.727+0000 7fd77f7fe640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:45.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.727+0000 7fd77f7fe640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.924+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fd778000b90 con 0x7fd7840399c0 2024-05-08T15:55:45.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.924+0000 7fd77f7fe640 1 -- 172.21.15.7:0/4186696458 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (unknown 3049115190 0 0) 0x7fd778000b90 con 0x7fd7840399c0 2024-05-08T15:55:45.920 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 >> v1:172.21.15.81:6812/847627118 conn(0x7fd7840399c0 legacy=0x7fd78403be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 10 monclient: shutdown 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 >> v1:172.21.15.7:6793/0 conn(0x7fd7980a5780 legacy=0x7fd79813c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 shutdown_connections 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 >> 172.21.15.7:0/4186696458 conn(0x7fd7980a9fa0 msgr2=0x7fd7980a7da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 shutdown_connections 2024-05-08T15:55:45.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:45.926+0000 7fd79d725640 1 -- 172.21.15.7:0/4186696458 wait complete. 2024-05-08T15:55:45.933 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T15:55:45.933 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T15:55:46.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.043+0000 7fa894be6640 10 monclient: get_monmap_and_config 2024-05-08T15:55:46.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.043+0000 7fa894be6640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.044+0000 7fa894be6640 10 monclient: monmap: 2024-05-08T15:55:46.039 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:46.044975+0000 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:46.044975+0000 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.040 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.041 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.044+0000 7fa894be6640 1 Processor -- start 2024-05-08T15:55:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.044+0000 7fa894be6640 1 -- start start 2024-05-08T15:55:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.044+0000 7fa894be6640 10 monclient: init 2024-05-08T15:55:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 10 monclient: _add_conns ranks=[5,6,2,0,7,3,8,1,4] 2024-05-08T15:55:46.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 10 monclient(hunting): picked mon.noname-h con 0x7fa8900a5250 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 10 monclient(hunting): picked mon.noname-d con 0x7fa8900a48c0 addr v1:172.21.15.7:6792/0 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 10 monclient(hunting): picked mon.noname-b con 0x7fa89013e340 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8900a28a0 con 0x7fa89013e340 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8900a2b80 con 0x7fa8900a5250 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 1 -- --> v1:172.21.15.7:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8900a2e60 con 0x7fa8900a48c0 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa894be6640 10 monclient(hunting): authenticate will time out at 1354.159916s 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88dd9b640 1 --1- >> v1:172.21.15.7:6792/0 conn(0x7fa8900a48c0 0x7fa89013cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6792/0 says I am v1:172.21.15.7:43878/0 (socket says 172.21.15.7:43878) 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88ed9d640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fa89013e340 0x7fa890140730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37068/0 (socket says 172.21.15.7:37068) 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88dd9b640 1 -- 172.21.15.7:0/1036075629 learned_addr learned my addr 172.21.15.7:0/1036075629 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 <== mon.6 v1:172.21.15.7:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3898520606 0 0) 0x7fa8900a2e60 con 0x7fa8900a48c0 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): my global_id is 124728 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 --> v1:172.21.15.7:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa878003520 con 0x7fa8900a48c0 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459397599 0 0) 0x7fa8900a28a0 con 0x7fa89013e340 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): my global_id is 96167 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8900a2e60 con 0x7fa89013e340 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4078714195 0 0) 0x7fa8900a2b80 con 0x7fa8900a5250 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 10 monclient(hunting): my global_id is 6755 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8900a28a0 con 0x7fa8900a5250 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.045+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 <== mon.6 v1:172.21.15.7:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 152147856 0 0) 0x7fa878003520 con 0x7fa8900a48c0 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 >> v1:172.21.15.81:6791/0 conn(0x7fa8900a5250 legacy=0x7fa8900a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 >> v1:172.21.15.7:6790/0 conn(0x7fa89013e340 legacy=0x7fa890140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 1 monclient: found mon.noname-d 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: _send_mon_message to mon.noname-d at v1:172.21.15.7:6792/0 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 --> v1:172.21.15.7:6792/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8900a42c0 con 0x7fa8900a48c0 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 <== mon.6 v1:172.21.15.7:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa8840017b0 con 0x7fa8900a48c0 2024-05-08T15:55:46.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 5 monclient: authenticate success, global_id 124728 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: got monmap 1 from mon.noname-d (according to old e1) 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: dump: 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.052 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 <== mon.6 v1:172.21.15.7:6792/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa884001a60 con 0x7fa8900a48c0 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 1 -- 172.21.15.7:0/1036075629 <== mon.6 v1:172.21.15.7:6792/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa884004be0 con 0x7fa8900a48c0 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: got monmap 1 from mon.d (according to old e1) 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa88d59a640 10 monclient: dump: 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.053 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 4 set_mon_vals no callback set 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 10 monclient: shutdown 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 1 -- 172.21.15.7:0/1036075629 >> v1:172.21.15.7:6792/0 conn(0x7fa8900a48c0 legacy=0x7fa89013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 1 -- 172.21.15.7:0/1036075629 shutdown_connections 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 1 -- 172.21.15.7:0/1036075629 >> 172.21.15.7:0/1036075629 conn(0x7fa8900a9ec0 msgr2=0x7fa8900aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 1 -- 172.21.15.7:0/1036075629 shutdown_connections 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.046+0000 7fa894be6640 1 -- 172.21.15.7:0/1036075629 wait complete. 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.047+0000 7fa894be6640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.047+0000 7fa894be6640 10 monclient: monmap: 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:46.055 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.047+0000 7fa894be6640 1 Processor -- start 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.047+0000 7fa894be6640 1 -- start start 2024-05-08T15:55:46.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.047+0000 7fa894be6640 10 monclient: init 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 10 monclient: _add_conns ranks=[0,3,7,2,5,4,6,8,1] 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 10 monclient(hunting): picked mon.noname-a con 0x7fa8900a48c0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 10 monclient(hunting): picked mon.noname-g con 0x7fa8900a5250 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 10 monclient(hunting): picked mon.noname-i con 0x7fa89013e340 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89013cf50 con 0x7fa8900a48c0 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89013c6f0 con 0x7fa8900a5250 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89013d230 con 0x7fa89013e340 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa894be6640 10 monclient(hunting): authenticate will time out at 1354.163001s 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa88e59c640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fa8900a48c0 0x7fa89012e8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41712/0 (socket says 172.21.15.7:41712) 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa88e59c640 1 -- 172.21.15.7:0/114288266 learned_addr learned my addr 172.21.15.7:0/114288266 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2274396805 0 0) 0x7fa89013cf50 con 0x7fa8900a48c0 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): my global_id is 95967 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa870003520 con 0x7fa8900a48c0 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2447814060 0 0) 0x7fa89013d230 con 0x7fa89013e340 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): my global_id is 115360 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa89013cf50 con 0x7fa89013e340 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537375966 0 0) 0x7fa89013c6f0 con 0x7fa8900a5250 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 10 monclient(hunting): my global_id is 77043 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa89013d230 con 0x7fa8900a5250 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.048+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1152705415 0 0) 0x7fa870003520 con 0x7fa8900a48c0 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa89013c6f0 con 0x7fa8900a48c0 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa8800018d0 con 0x7fa8900a48c0 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): dump: 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.059 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3801678325 0 0) 0x7fa89013cf50 con 0x7fa89013e340 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa870003520 con 0x7fa89013e340 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa8880018b0 con 0x7fa89013e340 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:46.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): dump: 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.061 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 671331820 0 0) 0x7fa89013d230 con 0x7fa8900a5250 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa89013cf50 con 0x7fa8900a5250 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa884004b20 con 0x7fa8900a5250 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient(hunting): dump: 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.062 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2611497852 0 0) 0x7fa89013c6f0 con 0x7fa8900a48c0 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 >> v1:172.21.15.81:6792/0 conn(0x7fa89013e340 legacy=0x7fa8901d5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 >> v1:172.21.15.81:6790/0 conn(0x7fa8900a5250 legacy=0x7fa8901d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 monclient: found mon.a 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8900a1ee0 con 0x7fa8900a48c0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa894be6640 5 monclient: authenticate success, global_id 95967 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa894be6640 10 monclient: _renew_subs 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa894be6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa8900a1b20 con 0x7fa8900a48c0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa894be6640 10 monclient: _renew_subs 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa894be6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fa89013b380 con 0x7fa8900a48c0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa880003c20 con 0x7fa8900a48c0 2024-05-08T15:55:46.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa880004190 con 0x7fa8900a48c0 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa88cd99640 4 set_mon_vals no callback set 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.049+0000 7fa87effd640 10 monclient: dump: 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.065 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa894be6640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.7:6789/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fa874000f80 con 0x7fa8900a48c0 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa880004330 con 0x7fa8900a48c0 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fa8800470b0 con 0x7fa8900a48c0 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.81:6802/1326537731 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa870044c20 con 0x7fa870040fb0 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_get_version_reply(handle=1 version=323) ==== 24+0+0 (unknown 4002016596 0 0) 0x7fa880047520 con 0x7fa8900a48c0 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa87effd640 10 monclient: handle_get_version_reply finishing 1 version 323 2024-05-08T15:55:46.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa88dd9b640 10 monclient: get_auth_request con 0x7fa87003a470 auth_method 0 2024-05-08T15:55:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.050+0000 7fa88ed9d640 10 monclient: get_auth_request con 0x7fa870040fb0 auth_method 0 2024-05-08T15:55:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.051+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== osd.3 v1:172.21.15.81:6802/1326537731 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fa870044c20 con 0x7fa870040fb0 2024-05-08T15:55:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.064+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 --> v1:172.21.15.81:6802/1326537731 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa874002cb0 con 0x7fa870040fb0 2024-05-08T15:55:46.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa87effd640 1 -- 172.21.15.7:0/114288266 <== osd.3 v1:172.21.15.81:6802/1326537731 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fa874002cb0 con 0x7fa870040fb0 2024-05-08T15:55:46.068 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:46.069 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T15:55:46.069 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 >> v1:172.21.15.81:6802/1326537731 conn(0x7fa870040fb0 legacy=0x7fa8700433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 >> v1:172.21.15.81:6812/847627118 conn(0x7fa87003a470 legacy=0x7fa87003c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 10 monclient: shutdown 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 >> v1:172.21.15.7:6789/0 conn(0x7fa8900a48c0 legacy=0x7fa89012e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa88ed9d640 1 -- 172.21.15.7:0/114288266 reap_dead start 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 shutdown_connections 2024-05-08T15:55:46.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 >> 172.21.15.7:0/114288266 conn(0x7fa8900a9ec0 msgr2=0x7fa8900755e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 shutdown_connections 2024-05-08T15:55:46.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.065+0000 7fa894be6640 1 -- 172.21.15.7:0/114288266 wait complete. 2024-05-08T15:55:46.072 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T15:55:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.180+0000 7f7a85ea1640 10 monclient: get_monmap_and_config 2024-05-08T15:55:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.180+0000 7f7a85ea1640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.180+0000 7f7a85ea1640 10 monclient: monmap: 2024-05-08T15:55:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.176 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:46.181870+0000 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:46.181870+0000 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.177 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.181+0000 7f7a85ea1640 1 Processor -- start 2024-05-08T15:55:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.181+0000 7f7a85ea1640 1 -- start start 2024-05-08T15:55:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.181+0000 7f7a85ea1640 10 monclient: init 2024-05-08T15:55:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 10 monclient: _add_conns ranks=[4,7,0,8,1,6,2,5,3] 2024-05-08T15:55:46.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 10 monclient(hunting): picked mon.noname-c con 0x7f7a80137940 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 10 monclient(hunting): picked mon.noname-i con 0x7f7a80138430 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 10 monclient(hunting): picked mon.noname-a con 0x7f7a800a49a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a800a2990 con 0x7f7a800a49a0 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a800a2c70 con 0x7f7a80137940 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a800a39f0 con 0x7f7a80138430 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 10 monclient(hunting): authenticate will time out at 1354.296865s 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7ffff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7a800a49a0 0x7f7a80140610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41714/0 (socket says 172.21.15.7:41714) 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7ffff640 1 -- 172.21.15.7:0/4188146221 learned_addr learned my addr 172.21.15.7:0/4188146221 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7f7fe640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f7a80137940 0x7f7a80137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:44990/0 (socket says 172.21.15.7:44990) 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126834541 0 0) 0x7f7a800a2c70 con 0x7f7a80137940 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): my global_id is 124537 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a68003520 con 0x7f7a80137940 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 832310741 0 0) 0x7f7a780013c0 con 0x7f7a800a49a0 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): my global_id is 95976 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a800a2c70 con 0x7f7a800a49a0 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3159902215 0 0) 0x7f7a800a39f0 con 0x7f7a80138430 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient(hunting): my global_id is 115369 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a780013c0 con 0x7f7a80138430 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2310275795 0 0) 0x7f7a800a2c70 con 0x7f7a800a49a0 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 >> v1:172.21.15.81:6792/0 conn(0x7f7a80138430 legacy=0x7f7a8013cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 >> v1:172.21.15.7:6791/0 conn(0x7f7a80137940 legacy=0x7f7a80137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 monclient: found mon.noname-a 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a800a4290 con 0x7f7a800a49a0 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a78001c10 con 0x7f7a800a49a0 2024-05-08T15:55:46.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a7e7fc640 10 monclient: dump: 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.189 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 5 monclient: authenticate success, global_id 95976 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.182+0000 7f7a85ea1640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7a78001f70 con 0x7f7a800a49a0 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a7e7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a7e7fc640 1 -- 172.21.15.7:0/4188146221 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a78004f60 con 0x7f7a800a49a0 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 4 set_mon_vals no callback set 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a7e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a7e7fc640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a7e7fc640 10 monclient: dump: 2024-05-08T15:55:46.190 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.191 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 10 monclient: shutdown 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 1 -- 172.21.15.7:0/4188146221 >> v1:172.21.15.7:6789/0 conn(0x7f7a800a49a0 legacy=0x7f7a80140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 1 -- 172.21.15.7:0/4188146221 shutdown_connections 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 1 -- 172.21.15.7:0/4188146221 >> 172.21.15.7:0/4188146221 conn(0x7f7a800a9fa0 msgr2=0x7f7a800aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 1 -- 172.21.15.7:0/4188146221 shutdown_connections 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.183+0000 7f7a85ea1640 1 -- 172.21.15.7:0/4188146221 wait complete. 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.184+0000 7f7a85ea1640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.184+0000 7f7a85ea1640 10 monclient: monmap: 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.192 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.184+0000 7f7a85ea1640 1 Processor -- start 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.184+0000 7f7a85ea1640 1 -- start start 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.184+0000 7f7a85ea1640 10 monclient: init 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient: _add_conns ranks=[3,0,7,4,2,5,6,8,1] 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient(hunting): picked mon.noname-g con 0x7f7a800a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient(hunting): picked mon.noname-a con 0x7f7a80137940 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient(hunting): picked mon.noname-i con 0x7f7a80138430 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a801cd220 con 0x7f7a80137940 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a800a0f60 con 0x7f7a800a49a0 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a800a1f30 con 0x7f7a80138430 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient(hunting): authenticate will time out at 1354.299815s 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a7effd640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f7a80137940 0x7f7a801d1980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41718/0 (socket says 172.21.15.7:41718) 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a7effd640 1 -- 172.21.15.7:0/825784779 learned_addr learned my addr 172.21.15.7:0/825784779 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1750178386 0 0) 0x7f7a780013c0 con 0x7f7a80137940 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): my global_id is 95985 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a6c003520 con 0x7f7a80137940 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3054237563 0 0) 0x7f7a800a0f60 con 0x7f7a800a49a0 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): my global_id is 77052 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a780013c0 con 0x7f7a800a49a0 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347907738 0 0) 0x7f7a800a1f30 con 0x7f7a80138430 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): my global_id is 115378 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a800a0f60 con 0x7f7a80138430 2024-05-08T15:55:46.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3797233252 0 0) 0x7f7a6c003520 con 0x7f7a80137940 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a800a1f30 con 0x7f7a80137940 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a74001a90 con 0x7f7a80137940 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): dump: 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.196 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.3 v1:172.21.15.81:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1982789156 0 0) 0x7f7a780013c0 con 0x7f7a800a49a0 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.81:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a6c003520 con 0x7f7a800a49a0 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.3 v1:172.21.15.81:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a70001c30 con 0x7f7a800a49a0 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): got monmap 1 from mon.g (according to old e1) 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient(hunting): dump: 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.197 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.198 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2402599132 0 0) 0x7f7a800a0f60 con 0x7f7a80138430 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f7a780013c0 con 0x7f7a80138430 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 942310922 0 0) 0x7f7a800a1f30 con 0x7f7a80137940 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 >> v1:172.21.15.81:6792/0 conn(0x7f7a80138430 legacy=0x7f7a801d50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 >> v1:172.21.15.81:6790/0 conn(0x7f7a800a49a0 legacy=0x7f7a80140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 monclient: found mon.a 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a80136780 con 0x7f7a80137940 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a67fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 5 monclient: authenticate success, global_id 95985 2024-05-08T15:55:46.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient: _renew_subs 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.185+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f7a80139190 con 0x7f7a80137940 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a85ea1640 10 monclient: _renew_subs 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a85ea1640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f7a80137500 con 0x7f7a80137940 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f7a74003940 con 0x7f7a80137940 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a67fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a7dffb640 4 set_mon_vals no callback set 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f7a74003e30 con 0x7f7a80137940 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a67fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a67fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a67fff640 10 monclient: dump: 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.200 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f7a74003fd0 con 0x7f7a80137940 2024-05-08T15:55:46.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a85ea1640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.186+0000 7f7a85ea1640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.187+0000 7f7a85ea1640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.187+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a60005080 con 0x7f7a80137940 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.187+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f7a74046c80 con 0x7f7a80137940 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.187+0000 7f7a7f7fe640 10 monclient: get_auth_request con 0x7f7a6c03a350 auth_method 0 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.191+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f7a801cd220 con 0x7f7a80137940 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.191+0000 7f7a67fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:46.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.191+0000 7f7a67fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:46.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.379+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f7a60000b90 con 0x7f7a6c03a350 2024-05-08T15:55:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.380+0000 7f7a67fff640 1 -- 172.21.15.7:0/825784779 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (unknown 3526570807 0 0) 0x7f7a60000b90 con 0x7f7a6c03a350 2024-05-08T15:55:46.376 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 >> v1:172.21.15.81:6812/847627118 conn(0x7f7a6c03a350 legacy=0x7f7a6c03c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 10 monclient: shutdown 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 >> v1:172.21.15.7:6789/0 conn(0x7f7a80137940 legacy=0x7f7a801d1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 shutdown_connections 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 >> 172.21.15.7:0/825784779 conn(0x7f7a800a9fa0 msgr2=0x7f7a800a7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 shutdown_connections 2024-05-08T15:55:46.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.382+0000 7f7a85ea1640 1 -- 172.21.15.7:0/825784779 wait complete. 2024-05-08T15:55:46.388 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T15:55:46.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T15:55:46.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.499+0000 7f0a352df640 10 monclient: get_monmap_and_config 2024-05-08T15:55:46.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.499+0000 7f0a352df640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.499+0000 7f0a352df640 10 monclient: monmap: 2024-05-08T15:55:46.495 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.495 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.495 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:46.500710+0000 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:46.500710+0000 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.496 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.500+0000 7f0a352df640 1 Processor -- start 2024-05-08T15:55:46.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.500+0000 7f0a352df640 1 -- start start 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.500+0000 7f0a352df640 10 monclient: init 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.500+0000 7f0a352df640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.500+0000 7f0a352df640 10 monclient: _add_conns ranks=[2,5,3,4,0,6,1,7,8] 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.500+0000 7f0a352df640 10 monclient(hunting): picked mon.noname-b con 0x7f0a300a5250 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 10 monclient(hunting): picked mon.noname-h con 0x7f0a300a48c0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 10 monclient(hunting): picked mon.noname-g con 0x7f0a3013e340 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a300a28a0 con 0x7f0a300a5250 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a300a2b80 con 0x7f0a3013e340 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a300a2e60 con 0x7f0a300a48c0 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 10 monclient(hunting): authenticate will time out at 1354.615661s 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2effd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f0a300a5250 0x7f0a300a5630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37078/0 (socket says 172.21.15.7:37078) 2024-05-08T15:55:46.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2effd640 1 -- 172.21.15.7:0/3212423777 learned_addr learned my addr 172.21.15.7:0/3212423777 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 354686690 0 0) 0x7f0a300a28a0 con 0x7f0a300a5250 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): my global_id is 96176 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a1c003520 con 0x7f0a300a5250 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3537767021 0 0) 0x7f0a300a2b80 con 0x7f0a3013e340 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient(hunting): my global_id is 77061 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a300a28a0 con 0x7f0a3013e340 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3241456720 0 0) 0x7f0a1c003520 con 0x7f0a300a5250 2024-05-08T15:55:46.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 >> v1:172.21.15.81:6791/0 conn(0x7f0a300a48c0 legacy=0x7f0a3013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 >> v1:172.21.15.81:6790/0 conn(0x7f0a3013e340 legacy=0x7f0a30140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 monclient: found mon.noname-b 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a300a42c0 con 0x7f0a300a5250 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0a200017b0 con 0x7f0a300a5250 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: dump: 2024-05-08T15:55:46.507 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.508 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 5 monclient: authenticate success, global_id 96176 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0a20001a60 con 0x7f0a300a5250 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 1 -- 172.21.15.7:0/3212423777 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0a20004be0 con 0x7f0a300a5250 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a352df640 4 set_mon_vals no callback set 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.501+0000 7f0a2e7fc640 10 monclient: dump: 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 10 monclient: shutdown 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 1 -- 172.21.15.7:0/3212423777 >> v1:172.21.15.7:6790/0 conn(0x7f0a300a5250 legacy=0x7f0a300a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 1 -- 172.21.15.7:0/3212423777 shutdown_connections 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 1 -- 172.21.15.7:0/3212423777 >> 172.21.15.7:0/3212423777 conn(0x7f0a300a9ec0 msgr2=0x7f0a300aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 1 -- 172.21.15.7:0/3212423777 shutdown_connections 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 1 -- 172.21.15.7:0/3212423777 wait complete. 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.502+0000 7f0a352df640 10 monclient: monmap: 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 1 Processor -- start 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 1 -- start start 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient: init 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient: _add_conns ranks=[8,4,7,5,0,2,1,3,6] 2024-05-08T15:55:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient(hunting): picked mon.noname-e con 0x7f0a300a48c0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient(hunting): picked mon.noname-c con 0x7f0a300a5250 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient(hunting): picked mon.noname-i con 0x7f0a3013e340 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a3013cbf0 con 0x7f0a300a5250 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a3013bc50 con 0x7f0a3013e340 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a3013bf30 con 0x7f0a300a48c0 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a352df640 10 monclient(hunting): authenticate will time out at 1354.618530s 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a2effd640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f0a300a48c0 0x7f0a301389f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55806/0 (socket says 172.21.15.7:55806) 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.503+0000 7f0a2effd640 1 -- 172.21.15.7:0/3678619604 learned_addr learned my addr 172.21.15.7:0/3678619604 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4212254678 0 0) 0x7f0a3013bf30 con 0x7f0a300a48c0 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): my global_id is 144251 2024-05-08T15:55:46.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a10003520 con 0x7f0a300a48c0 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1728753858 0 0) 0x7f0a18001230 con 0x7f0a300a5250 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): my global_id is 124546 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a3013bf30 con 0x7f0a300a5250 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303795824 0 0) 0x7f0a3013bc50 con 0x7f0a3013e340 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): my global_id is 115387 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a18001230 con 0x7f0a3013e340 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1680462559 0 0) 0x7f0a10003520 con 0x7f0a300a48c0 2024-05-08T15:55:46.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0a3013bc50 con 0x7f0a300a48c0 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0a20004b20 con 0x7f0a300a48c0 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): dump: 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.515 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 894525062 0 0) 0x7f0a3013bf30 con 0x7f0a300a5250 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0a10003520 con 0x7f0a300a5250 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0a18001ca0 con 0x7f0a300a5250 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): dump: 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.516 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 151394915 0 0) 0x7f0a18001230 con 0x7f0a3013e340 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f0a3013bf30 con 0x7f0a3013e340 2024-05-08T15:55:46.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0a28001cf0 con 0x7f0a3013e340 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient(hunting): dump: 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.518 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 2401532249 0 0) 0x7f0a3013bc50 con 0x7f0a300a48c0 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 >> v1:172.21.15.81:6792/0 conn(0x7f0a3013e340 legacy=0x7f0a301d4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 >> v1:172.21.15.7:6791/0 conn(0x7f0a300a5250 legacy=0x7f0a30139100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 monclient: found mon.e 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a30139810 con 0x7f0a300a48c0 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a352df640 5 monclient: authenticate success, global_id 144251 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a352df640 10 monclient: _renew_subs 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a352df640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3013b6d0 con 0x7f0a300a48c0 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a352df640 10 monclient: _renew_subs 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a352df640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f0a300a1b20 con 0x7f0a300a48c0 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0a20001a60 con 0x7f0a300a48c0 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0a20001790 con 0x7f0a300a48c0 2024-05-08T15:55:46.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a2dffb640 4 set_mon_vals no callback set 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.504+0000 7f0a277fe640 10 monclient: dump: 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.521 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.505+0000 7f0a352df640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.505+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.7:6793/0 -- mon_get_version(what=osdmap handle=1) -- 0x7f0a14000f80 con 0x7f0a300a48c0 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.505+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f0a20010410 con 0x7f0a300a48c0 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.505+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f0a20046e50 con 0x7f0a300a48c0 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.505+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.81:6808/193838907 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a10044bf0 con 0x7f0a10040f80 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.505+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_get_version_reply(handle=1 version=323) ==== 24+0+0 (unknown 4002016596 0 0) 0x7f0a200472c0 con 0x7f0a300a48c0 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.505+0000 7f0a277fe640 10 monclient: handle_get_version_reply finishing 1 version 323 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.506+0000 7f0a27fff640 10 monclient: get_auth_request con 0x7f0a1003a3a0 auth_method 0 2024-05-08T15:55:46.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.506+0000 7f0a2f7fe640 10 monclient: get_auth_request con 0x7f0a10040f80 auth_method 0 2024-05-08T15:55:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.507+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== osd.4 v1:172.21.15.81:6808/193838907 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f0a10044bf0 con 0x7f0a10040f80 2024-05-08T15:55:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.520+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 --> v1:172.21.15.81:6808/193838907 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0a14002cb0 con 0x7f0a10040f80 2024-05-08T15:55:46.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.520+0000 7f0a277fe640 1 -- 172.21.15.7:0/3678619604 <== osd.4 v1:172.21.15.81:6808/193838907 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7f0a14002cb0 con 0x7f0a10040f80 2024-05-08T15:55:46.524 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:46.524 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T15:55:46.524 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 >> v1:172.21.15.81:6808/193838907 conn(0x7f0a10040f80 legacy=0x7f0a100433a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 >> v1:172.21.15.81:6812/847627118 conn(0x7f0a1003a3a0 legacy=0x7f0a1003c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 10 monclient: shutdown 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 >> v1:172.21.15.7:6793/0 conn(0x7f0a300a48c0 legacy=0x7f0a301389f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a2f7fe640 1 -- 172.21.15.7:0/3678619604 reap_dead start 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 shutdown_connections 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 >> 172.21.15.7:0/3678619604 conn(0x7f0a300a9ec0 msgr2=0x7f0a30135f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 shutdown_connections 2024-05-08T15:55:46.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.521+0000 7f0a352df640 1 -- 172.21.15.7:0/3678619604 wait complete. 2024-05-08T15:55:46.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.635+0000 7f0460179640 10 monclient: get_monmap_and_config 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.635+0000 7f0460179640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.635+0000 7f0460179640 10 monclient: monmap: 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:46.636637+0000 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:46.636637+0000 2024-05-08T15:55:46.631 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.632 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.636+0000 7f0460179640 1 Processor -- start 2024-05-08T15:55:46.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.636+0000 7f0460179640 1 -- start start 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.636+0000 7f0460179640 10 monclient: init 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.636+0000 7f0460179640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.636+0000 7f0460179640 10 monclient: _add_conns ranks=[5,7,4,8,2,3,0,1,6] 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.636+0000 7f0460179640 10 monclient(hunting): picked mon.noname-h con 0x7f04580a5430 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 10 monclient(hunting): picked mon.noname-i con 0x7f04580a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 10 monclient(hunting): picked mon.noname-c con 0x7f045813e380 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04580a2990 con 0x7f045813e380 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04580a2c70 con 0x7f04580a5430 2024-05-08T15:55:46.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04580a2f50 con 0x7f04580a49a0 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 10 monclient(hunting): authenticate will time out at 1354.751709s 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045e716640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f045813e380 0x7f0458140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:45004/0 (socket says 172.21.15.7:45004) 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045e716640 1 -- 172.21.15.7:0/183348587 learned_addr learned my addr 172.21.15.7:0/183348587 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209447813 0 0) 0x7f04580a2990 con 0x7f045813e380 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): my global_id is 124555 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0444003520 con 0x7f045813e380 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517899296 0 0) 0x7f04580a2f50 con 0x7f04580a49a0 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): my global_id is 115396 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04580a2990 con 0x7f04580a49a0 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 197739265 0 0) 0x7f04580a2c70 con 0x7f04580a5430 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient(hunting): my global_id is 6773 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04580a2f50 con 0x7f04580a5430 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1752328581 0 0) 0x7f0444003520 con 0x7f045813e380 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 >> v1:172.21.15.81:6792/0 conn(0x7f04580a49a0 legacy=0x7f045813cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 >> v1:172.21.15.81:6791/0 conn(0x7f04580a5430 legacy=0x7f04580a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 monclient: found mon.noname-c 2024-05-08T15:55:46.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: _send_mon_message to mon.noname-c at v1:172.21.15.7:6791/0 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 --> v1:172.21.15.7:6791/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04580a4300 con 0x7f045813e380 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0450001b30 con 0x7f045813e380 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: got monmap 1 from mon.noname-c (according to old e1) 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: dump: 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.643 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 5 monclient: authenticate success, global_id 124555 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 <== mon.4 v1:172.21.15.7:6791/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0450001e50 con 0x7f045813e380 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 1 -- 172.21.15.7:0/183348587 <== mon.4 v1:172.21.15.7:6791/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0450004f60 con 0x7f045813e380 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f045cf13640 10 monclient: dump: 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.645 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.637+0000 7f0460179640 4 set_mon_vals no callback set 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.638+0000 7f0460179640 10 monclient: shutdown 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.638+0000 7f0460179640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.638+0000 7f0460179640 1 -- 172.21.15.7:0/183348587 >> v1:172.21.15.7:6791/0 conn(0x7f045813e380 legacy=0x7f0458140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.638+0000 7f0460179640 1 -- 172.21.15.7:0/183348587 shutdown_connections 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.638+0000 7f0460179640 1 -- 172.21.15.7:0/183348587 >> 172.21.15.7:0/183348587 conn(0x7f04580a9fa0 msgr2=0x7f04580aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.638+0000 7f0460179640 1 -- 172.21.15.7:0/183348587 shutdown_connections 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.638+0000 7f0460179640 1 -- 172.21.15.7:0/183348587 wait complete. 2024-05-08T15:55:46.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient: monmap: 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.647 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 1 Processor -- start 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 1 -- start start 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient: init 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient: _add_conns ranks=[5,1,7,0,8,3,2,4,6] 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient(hunting): picked mon.noname-h con 0x7f04580a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient(hunting): picked mon.noname-f con 0x7f04580a5430 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient(hunting): picked mon.noname-i con 0x7f045813e380 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04580a0f60 con 0x7f04580a5430 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04581346a0 con 0x7f04580a49a0 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04581360f0 con 0x7f045813e380 2024-05-08T15:55:46.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.639+0000 7f0460179640 10 monclient(hunting): authenticate will time out at 1354.754543s 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f045df15640 1 --1- >> v1:172.21.15.81:6791/0 conn(0x7f04580a49a0 0x7f04581cc540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6791/0 says I am v1:172.21.15.7:39482/0 (socket says 172.21.15.7:39482) 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f045d714640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f04580a5430 0x7f0458137ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.7:34108/0 (socket says 172.21.15.7:34108) 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f045e716640 1 --1- >> v1:172.21.15.81:6792/0 conn(0x7f045813e380 0x7f04581d4880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.81:6792/0 says I am v1:172.21.15.7:46484/0 (socket says 172.21.15.7:46484) 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f045df15640 1 -- 172.21.15.7:0/2858880325 learned_addr learned my addr 172.21.15.7:0/2858880325 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 454329004 0 0) 0x7f04580a2990 con 0x7f04580a5430 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): my global_id is 6616 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f043c003520 con 0x7f04580a5430 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312147057 0 0) 0x7f04581360f0 con 0x7f045813e380 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): my global_id is 115405 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04580a2990 con 0x7f045813e380 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143380900 0 0) 0x7f04581346a0 con 0x7f04580a49a0 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): my global_id is 6782 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04581360f0 con 0x7f04580a49a0 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2083269441 0 0) 0x7f043c003520 con 0x7f04580a5430 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f04581346a0 con 0x7f04580a5430 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f04540018d0 con 0x7f04580a5430 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-f (according to old e1) 2024-05-08T15:55:46.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.651 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1773053406 0 0) 0x7f04581360f0 con 0x7f04580a49a0 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f043c003520 con 0x7f04580a49a0 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.7 v1:172.21.15.81:6792/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2732970130 0 0) 0x7f04580a2990 con 0x7f045813e380 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6792/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f04581360f0 con 0x7f045813e380 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f044c001a10 con 0x7f04580a49a0 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.652 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.7 v1:172.21.15.81:6792/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0450004ea0 con 0x7f045813e380 2024-05-08T15:55:46.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): got monmap 1 from mon.i (according to old e1) 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.654 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 4023445503 0 0) 0x7f04581346a0 con 0x7f04580a5430 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 >> v1:172.21.15.81:6792/0 conn(0x7f045813e380 legacy=0x7f04581d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 >> v1:172.21.15.81:6791/0 conn(0x7f04580a49a0 legacy=0x7f04581cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 monclient: found mon.f 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04581385d0 con 0x7f04580a5430 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f044a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f0460179640 5 monclient: authenticate success, global_id 6616 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f0460179640 10 monclient: _renew_subs 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f0460179640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.640+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f0458135200 con 0x7f04580a5430 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f0460179640 10 monclient: _renew_subs 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f0460179640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7f04581d5fa0 con 0x7f04580a5430 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f0454003610 con 0x7f04580a5430 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f0454003b80 con 0x7f04580a5430 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044bfff640 4 set_mon_vals no callback set 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044a7fc640 10 monclient: got monmap 1 from mon.f (according to old e1) 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044a7fc640 10 monclient: dump: 2024-05-08T15:55:46.656 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.657 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f0460179640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f0460179640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f0460179640 10 monclient: _send_mon_message to mon.f at v1:172.21.15.81:6789/0 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0440005080 con 0x7f04580a5430 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.641+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f0454003da0 con 0x7f04580a5430 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.642+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f0454046c70 con 0x7f04580a5430 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.642+0000 7f045df15640 10 monclient: get_auth_request con 0x7f043c03a530 auth_method 0 2024-05-08T15:55:46.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.647+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mon.1 v1:172.21.15.81:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f04580a0f60 con 0x7f04580a5430 2024-05-08T15:55:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.647+0000 7f044a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:46.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.647+0000 7f044a7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:46.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.836+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f0440000b90 con 0x7f043c03a530 2024-05-08T15:55:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.836+0000 7f044a7fc640 1 -- 172.21.15.7:0/2858880325 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (unknown 3782719169 0 0) 0x7f0440000b90 con 0x7f043c03a530 2024-05-08T15:55:46.832 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T15:55:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.838+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 >> v1:172.21.15.81:6812/847627118 conn(0x7f043c03a530 legacy=0x7f043c03c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.838+0000 7f0460179640 10 monclient: shutdown 2024-05-08T15:55:46.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.838+0000 7f0460179640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.838+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 >> v1:172.21.15.81:6789/0 conn(0x7f04580a5430 legacy=0x7f0458137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.838+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 shutdown_connections 2024-05-08T15:55:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.838+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 >> 172.21.15.7:0/2858880325 conn(0x7f04580a9fa0 msgr2=0x7f04580a7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.839+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 shutdown_connections 2024-05-08T15:55:46.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.839+0000 7f0460179640 1 -- 172.21.15.7:0/2858880325 wait complete. 2024-05-08T15:55:46.845 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T15:55:46.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T15:55:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.956+0000 7fd515b19640 10 monclient: get_monmap_and_config 2024-05-08T15:55:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.956+0000 7fd515b19640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.956+0000 7fd515b19640 10 monclient: monmap: 2024-05-08T15:55:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.952 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:46.957833+0000 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:46.957833+0000 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.953 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.957+0000 7fd515b19640 1 Processor -- start 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.957+0000 7fd515b19640 1 -- start start 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.957+0000 7fd515b19640 10 monclient: init 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 10 monclient: _add_conns ranks=[3,2,8,1,4,6,7,0,5] 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 10 monclient(hunting): picked mon.noname-g con 0x7fd5100a5250 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 10 monclient(hunting): picked mon.noname-b con 0x7fd5100a48c0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 10 monclient(hunting): picked mon.noname-e con 0x7fd51013e340 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5100a28a0 con 0x7fd5100a48c0 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5100a2b80 con 0x7fd5100a5250 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5100a2e60 con 0x7fd51013e340 2024-05-08T15:55:46.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 10 monclient(hunting): authenticate will time out at 1355.072805s 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50ffff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fd51013e340 0x7fd510140730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55808/0 (socket says 172.21.15.7:55808) 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50ffff640 1 -- 172.21.15.7:0/2222871822 learned_addr learned my addr 172.21.15.7:0/2222871822 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50effd640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fd5100a48c0 0x7fd51013cc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37086/0 (socket says 172.21.15.7:37086) 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039772137 0 0) 0x7fd5100a2e60 con 0x7fd51013e340 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): my global_id is 144260 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4f8003520 con 0x7fd51013e340 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2802581921 0 0) 0x7fd5100a28a0 con 0x7fd5100a48c0 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): my global_id is 96185 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5100a2e60 con 0x7fd5100a48c0 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3021837516 0 0) 0x7fd5100a2b80 con 0x7fd5100a5250 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient(hunting): my global_id is 77070 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5100a28a0 con 0x7fd5100a5250 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3148941529 0 0) 0x7fd4f8003520 con 0x7fd51013e340 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 >> v1:172.21.15.81:6790/0 conn(0x7fd5100a5250 legacy=0x7fd5100a5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 >> v1:172.21.15.7:6790/0 conn(0x7fd5100a48c0 legacy=0x7fd51013cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 monclient: found mon.noname-e 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: _send_mon_message to mon.noname-e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5100a42c0 con 0x7fd51013e340 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd508001b30 con 0x7fd51013e340 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 5 monclient: authenticate success, global_id 144260 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: dump: 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.964 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd515b19640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:46.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 <== mon.8 v1:172.21.15.7:6793/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd508001de0 con 0x7fd51013e340 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 1 -- 172.21.15.7:0/2222871822 <== mon.8 v1:172.21.15.7:6793/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd508004f60 con 0x7fd51013e340 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.958+0000 7fd50e7fc640 10 monclient: dump: 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.966 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 4 set_mon_vals no callback set 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 10 monclient: shutdown 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 1 -- 172.21.15.7:0/2222871822 >> v1:172.21.15.7:6793/0 conn(0x7fd51013e340 legacy=0x7fd510140730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 1 -- 172.21.15.7:0/2222871822 shutdown_connections 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 1 -- 172.21.15.7:0/2222871822 >> 172.21.15.7:0/2222871822 conn(0x7fd5100a9ec0 msgr2=0x7fd5100aa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 1 -- 172.21.15.7:0/2222871822 shutdown_connections 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.959+0000 7fd515b19640 1 -- 172.21.15.7:0/2222871822 wait complete. 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.960+0000 7fd515b19640 10 monclient: build_initial_monmap 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.960+0000 7fd515b19640 10 monclient: monmap: 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:46.968 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.960+0000 7fd515b19640 1 Processor -- start 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.960+0000 7fd515b19640 1 -- start start 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.960+0000 7fd515b19640 10 monclient: init 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 10 monclient: _add_conns ranks=[5,8,0,4,1,7,6,2,3] 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 10 monclient(hunting): picked mon.noname-h con 0x7fd5100a48c0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 10 monclient(hunting): picked mon.noname-e con 0x7fd5100a5250 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 10 monclient(hunting): picked mon.noname-a con 0x7fd51013e340 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:46.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd510138ce0 con 0x7fd51013e340 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd51013a200 con 0x7fd5100a48c0 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd510138a60 con 0x7fd5100a5250 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd515b19640 10 monclient(hunting): authenticate will time out at 1355.075773s 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd50effd640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7fd5100a5250 0x7fd5101d18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:55812/0 (socket says 172.21.15.7:55812) 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd50effd640 1 -- 172.21.15.7:0/1582237176 learned_addr learned my addr 172.21.15.7:0/1582237176 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3231013088 0 0) 0x7fd510138a60 con 0x7fd5100a5250 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): my global_id is 144269 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4fc003520 con 0x7fd5100a5250 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2121355089 0 0) 0x7fd5100a42c0 con 0x7fd51013e340 2024-05-08T15:55:46.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): my global_id is 95994 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd510138a60 con 0x7fd51013e340 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1352089802 0 0) 0x7fd51013a200 con 0x7fd5100a48c0 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): my global_id is 6791 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5100a42c0 con 0x7fd5100a48c0 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3284812386 0 0) 0x7fd4fc003520 con 0x7fd5100a5250 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd51013a200 con 0x7fd5100a5250 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd504001a50 con 0x7fd5100a5250 2024-05-08T15:55:46.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): dump: 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.972 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 187729156 0 0) 0x7fd510138a60 con 0x7fd51013e340 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd4fc003520 con 0x7fd51013e340 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd508004ea0 con 0x7fd51013e340 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): dump: 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.973 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2308583766 0 0) 0x7fd5100a42c0 con 0x7fd5100a48c0 2024-05-08T15:55:46.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fd510138a60 con 0x7fd5100a48c0 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd500001990 con 0x7fd5100a48c0 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient(hunting): dump: 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.975 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3345879931 0 0) 0x7fd51013a200 con 0x7fd5100a5250 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 >> v1:172.21.15.81:6791/0 conn(0x7fd5100a48c0 legacy=0x7fd51013ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 >> v1:172.21.15.7:6789/0 conn(0x7fd51013e340 legacy=0x7fd5101d4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 monclient: found mon.e 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5100a1ee0 con 0x7fd5100a5250 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient: _finish_auth 0 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.961+0000 7fd4f7fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 5 monclient: authenticate success, global_id 144269 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 10 monclient: _renew_subs 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fd5100a1b20 con 0x7fd5100a5250 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 10 monclient: _renew_subs 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fd510139f20 con 0x7fd5100a5250 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fd504004a90 con 0x7fd5100a5250 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd4f7fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fd504005000 con 0x7fd5100a5250 2024-05-08T15:55:46.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd4f7fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd50dffb640 4 set_mon_vals no callback set 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd4f7fff640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd4f7fff640 10 monclient: dump: 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:46.978 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fd5040051a0 con 0x7fd5100a5250 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.962+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.7:6793/0 -- mon_get_version(what=osdmap handle=1) -- 0x7fd4f0000f80 con 0x7fd5100a5250 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.963+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fd504046c50 con 0x7fd5100a5250 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.963+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.81:6800/385867939 -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd4fc044d90 con 0x7fd4fc041120 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.963+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_get_version_reply(handle=1 version=323) ==== 24+0+0 (unknown 4002016596 0 0) 0x7fd5040470c0 con 0x7fd5100a5250 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.963+0000 7fd4f7fff640 10 monclient: handle_get_version_reply finishing 1 version 323 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.963+0000 7fd50f7fe640 10 monclient: get_auth_request con 0x7fd4fc03a540 auth_method 0 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.963+0000 7fd50ffff640 10 monclient: get_auth_request con 0x7fd4fc041120 auth_method 0 2024-05-08T15:55:46.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.964+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== osd.5 v1:172.21.15.81:6800/385867939 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fd4fc044d90 con 0x7fd4fc041120 2024-05-08T15:55:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.977+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 --> v1:172.21.15.81:6800/385867939 -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd4f0002cb0 con 0x7fd4fc041120 2024-05-08T15:55:46.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.977+0000 7fd4f7fff640 1 -- 172.21.15.7:0/1582237176 <== osd.5 v1:172.21.15.81:6800/385867939 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (unknown 0 0 119112422) 0x7fd4f0002cb0 con 0x7fd4fc041120 2024-05-08T15:55:46.980 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-05-08T15:55:46.980 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T15:55:46.981 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-05-08T15:55:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 >> v1:172.21.15.81:6800/385867939 conn(0x7fd4fc041120 legacy=0x7fd4fc043540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 >> v1:172.21.15.81:6812/847627118 conn(0x7fd4fc03a540 legacy=0x7fd4fc03ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 10 monclient: shutdown 2024-05-08T15:55:46.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:46.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 >> v1:172.21.15.7:6793/0 conn(0x7fd5100a5250 legacy=0x7fd5101d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:46.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd50ffff640 1 -- 172.21.15.7:0/1582237176 reap_dead start 2024-05-08T15:55:46.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 shutdown_connections 2024-05-08T15:55:46.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 >> 172.21.15.7:0/1582237176 conn(0x7fd5100a9ec0 msgr2=0x7fd510136220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:46.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 shutdown_connections 2024-05-08T15:55:46.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:46.978+0000 7fd515b19640 1 -- 172.21.15.7:0/1582237176 wait complete. 2024-05-08T15:55:46.984 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T15:55:47.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.092+0000 7fe943872640 10 monclient: get_monmap_and_config 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.092+0000 7fe943872640 10 monclient: build_initial_monmap 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.092+0000 7fe943872640 10 monclient: monmap: 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:47.093325+0000 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:47.093325+0000 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:47.088 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:47.089 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:47.089 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.092+0000 7fe943872640 1 Processor -- start 2024-05-08T15:55:47.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 1 -- start start 2024-05-08T15:55:47.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient: init 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient: _add_conns ranks=[1,7,0,3,5,2,4,6,8] 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient(hunting): picked mon.noname-f con 0x7fe93c0a53a0 addr v1:172.21.15.81:6789/0 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient(hunting): picked mon.noname-i con 0x7fe93c0a5780 addr v1:172.21.15.81:6792/0 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient(hunting): picked mon.noname-a con 0x7fe93c13e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c0a1660 con 0x7fe93c13e310 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c0a28f0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c0a2bd0 con 0x7fe93c0a5780 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe943872640 10 monclient(hunting): authenticate will time out at 1355.208432s 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe941e0f640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe93c13e310 0x7fe93c140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41738/0 (socket says 172.21.15.7:41738) 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.093+0000 7fe941e0f640 1 -- 172.21.15.7:0/1041438714 learned_addr learned my addr 172.21.15.7:0/1041438714 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3814366559 0 0) 0x7fe93c0a1660 con 0x7fe93c13e310 2024-05-08T15:55:47.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient(hunting): my global_id is 96003 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe928003520 con 0x7fe93c13e310 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 168268339 0 0) 0x7fe928003520 con 0x7fe93c13e310 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 >> v1:172.21.15.81:6792/0 conn(0x7fe93c0a5780 legacy=0x7fe93c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 >> v1:172.21.15.81:6789/0 conn(0x7fe93c0a53a0 legacy=0x7fe93c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: _finish_hunting 0 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 monclient: found mon.noname-a 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: _send_mon_message to mon.noname-a at v1:172.21.15.7:6789/0 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c0a6730 con 0x7fe93c13e310 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: _finish_auth 0 2024-05-08T15:55:47.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: _check_auth_tickets 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe943872640 5 monclient: authenticate success, global_id 96003 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe934001b30 con 0x7fe93c13e310 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe943872640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: dump: 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.101 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe943872640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 <== mon.0 v1:172.21.15.7:6789/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe934001e90 con 0x7fe93c13e310 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 1 -- 172.21.15.7:0/1041438714 <== mon.0 v1:172.21.15.7:6789/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe934004f60 con 0x7fe93c13e310 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:47.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe933fff640 10 monclient: dump: 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.103 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe943872640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.094+0000 7fe943872640 4 set_mon_vals no callback set 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 10 monclient: shutdown 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 1 -- 172.21.15.7:0/1041438714 >> v1:172.21.15.7:6789/0 conn(0x7fe93c13e310 legacy=0x7fe93c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 1 -- 172.21.15.7:0/1041438714 shutdown_connections 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 1 -- 172.21.15.7:0/1041438714 >> 172.21.15.7:0/1041438714 conn(0x7fe93c0a9fa0 msgr2=0x7fe93c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 1 -- 172.21.15.7:0/1041438714 shutdown_connections 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 1 -- 172.21.15.7:0/1041438714 wait complete. 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 10 monclient: build_initial_monmap 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.095+0000 7fe943872640 10 monclient: monmap: 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:47.104 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:47.105 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 1 Processor -- start 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 1 -- start start 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient: init 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient: _add_conns ranks=[0,4,2,5,3,7,8,6,1] 2024-05-08T15:55:47.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient(hunting): picked mon.noname-a con 0x7fe93c0a53a0 addr v1:172.21.15.7:6789/0 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient(hunting): picked mon.noname-c con 0x7fe93c0a5780 addr v1:172.21.15.7:6791/0 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient(hunting): picked mon.noname-b con 0x7fe93c13e310 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c1379d0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c137cb0 con 0x7fe93c13e310 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe93c0a1e00 con 0x7fe93c0a5780 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe943872640 10 monclient(hunting): authenticate will time out at 1355.211195s 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe94160e640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7fe93c0a53a0 0x7fe93c133e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:41754/0 (socket says 172.21.15.7:41754) 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe940e0d640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7fe93c0a5780 0x7fe93c134520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:45020/0 (socket says 172.21.15.7:45020) 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe94160e640 1 -- 172.21.15.7:0/1132025430 learned_addr learned my addr 172.21.15.7:0/1132025430 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2645151356 0 0) 0x7fe93c1379d0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe931ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe931ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe931ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe931ffb640 10 monclient(hunting): my global_id is 96012 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.096+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe920003520 con 0x7fe93c0a53a0 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2803128142 0 0) 0x7fe93c0a1e00 con 0x7fe93c0a5780 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): my global_id is 124564 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe93c1379d0 con 0x7fe93c0a5780 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701973571 0 0) 0x7fe93c137cb0 con 0x7fe93c13e310 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): my global_id is 96194 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe93c0a1e00 con 0x7fe93c13e310 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1467031242 0 0) 0x7fe920003520 con 0x7fe93c0a53a0 2024-05-08T15:55:47.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe93c137cb0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe938001ab0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): got monmap 1 from mon.noname-a (according to old e1) 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.109 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.4 v1:172.21.15.7:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 2526860790 0 0) 0x7fe93c1379d0 con 0x7fe93c0a5780 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe920003520 con 0x7fe93c0a5780 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.4 v1:172.21.15.7:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe92c001910 con 0x7fe93c0a5780 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): got monmap 1 from mon.c (according to old e1) 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.110 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1287944388 0 0) 0x7fe93c0a1e00 con 0x7fe93c13e310 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fe93c1379d0 con 0x7fe93c13e310 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe934001af0 con 0x7fe93c13e310 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient(hunting): dump: 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.112 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 795348120 0 0) 0x7fe93c137cb0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 >> v1:172.21.15.7:6791/0 conn(0x7fe93c0a5780 legacy=0x7fe93c134520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 >> v1:172.21.15.7:6790/0 conn(0x7fe93c13e310 legacy=0x7fe93c1d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: _finish_hunting 0 2024-05-08T15:55:47.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 monclient: found mon.a 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c0a43c0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: _finish_auth 0 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: _check_auth_tickets 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe943872640 5 monclient: authenticate success, global_id 96012 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe943872640 10 monclient: _renew_subs 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe943872640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6789/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c134c30 con 0x7fe93c0a53a0 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fe9380038c0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fe938003e30 con 0x7fe93c0a53a0 2024-05-08T15:55:47.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: got monmap 1 from mon.a (according to old e1) 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe931ffb640 10 monclient: dump: 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.115 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe943872640 10 monclient: _renew_subs 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe9337fe640 4 set_mon_vals no callback set 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe943872640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.097+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6789/0 -- mon_subscribe({osdmap=0}) -- 0x7fe93c136270 con 0x7fe93c0a53a0 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.098+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fe938003fd0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.098+0000 7fe943872640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.098+0000 7fe943872640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.098+0000 7fe943872640 10 monclient: _send_mon_message to mon.a at v1:172.21.15.7:6789/0 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.098+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.7:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93c075820 con 0x7fe93c0a53a0 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.098+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fe938046da0 con 0x7fe93c0a53a0 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.099+0000 7fe940e0d640 10 monclient: get_auth_request con 0x7fe92003a590 auth_method 0 2024-05-08T15:55:47.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.103+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mon.0 v1:172.21.15.7:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fe938015f90 con 0x7fe93c0a53a0 2024-05-08T15:55:47.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.103+0000 7fe931ffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:47.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.103+0000 7fe931ffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:47.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.285+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fe93c1416b0 con 0x7fe92003a590 2024-05-08T15:55:47.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.285+0000 7fe931ffb640 1 -- 172.21.15.7:0/1132025430 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (unknown 2264484800 0 0) 0x7fe93c1416b0 con 0x7fe92003a590 2024-05-08T15:55:47.281 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T15:55:47.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.287+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 >> v1:172.21.15.81:6812/847627118 conn(0x7fe92003a590 legacy=0x7fe92003ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.287+0000 7fe943872640 10 monclient: shutdown 2024-05-08T15:55:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.287+0000 7fe943872640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.288+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 >> v1:172.21.15.7:6789/0 conn(0x7fe93c0a53a0 legacy=0x7fe93c133e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.288+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 shutdown_connections 2024-05-08T15:55:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.288+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 >> 172.21.15.7:0/1132025430 conn(0x7fe93c0a9fa0 msgr2=0x7fe93c13f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.288+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 shutdown_connections 2024-05-08T15:55:47.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.288+0000 7fe943872640 1 -- 172.21.15.7:0/1132025430 wait complete. 2024-05-08T15:55:47.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:55:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.403+0000 7fa23c04e640 10 monclient: get_monmap_and_config 2024-05-08T15:55:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.403+0000 7fa23c04e640 10 monclient: build_initial_monmap 2024-05-08T15:55:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.403+0000 7fa23c04e640 10 monclient: monmap: 2024-05-08T15:55:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:55:47.404606+0000 2024-05-08T15:55:47.399 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:55:47.404606+0000 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:47.400 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 1 Processor -- start 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 1 -- start start 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 10 monclient: init 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 10 monclient: _add_conns ranks=[2,3,5,1,6,4,7,8,0] 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 10 monclient(hunting): picked mon.noname-b con 0x7fa234137940 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 10 monclient(hunting): picked mon.noname-g con 0x7fa2340a49a0 addr v1:172.21.15.81:6790/0 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 10 monclient(hunting): picked mon.noname-h con 0x7fa2340a41b0 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.404+0000 7fa23c04e640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340a2c70 con 0x7fa234137940 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340a2f50 con 0x7fa2340a49a0 2024-05-08T15:55:47.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340a1670 con 0x7fa2340a41b0 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 10 monclient(hunting): authenticate will time out at 1355.519630s 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa239dea640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fa234137940 0x7fa234137d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37108/0 (socket says 172.21.15.7:37108) 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa239dea640 1 -- 172.21.15.7:0/4271437679 learned_addr learned my addr 172.21.15.7:0/4271437679 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1241921792 0 0) 0x7fa2340a2c70 con 0x7fa234137940 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient(hunting): my global_id is 96203 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa220003520 con 0x7fa234137940 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2944289593 0 0) 0x7fa220003520 con 0x7fa234137940 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 >> v1:172.21.15.81:6791/0 conn(0x7fa2340a41b0 legacy=0x7fa234140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 >> v1:172.21.15.81:6790/0 conn(0x7fa2340a49a0 legacy=0x7fa23413cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: _finish_hunting 0 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 monclient: found mon.noname-b 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2340a39f0 con 0x7fa234137940 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: _finish_auth 0 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: _check_auth_tickets 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 5 monclient: authenticate success, global_id 96203 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa2300017b0 con 0x7fa234137940 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:55:47.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: dump: 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.411 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa230001aa0 con 0x7fa234137940 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 10 monclient: get_monmap_and_config success 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa23c04e640 4 set_mon_vals no callback set 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 1 -- 172.21.15.7:0/4271437679 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa230004be0 con 0x7fa234137940 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.405+0000 7fa238de8640 10 monclient: dump: 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.412 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 10 monclient: shutdown 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 1 -- 172.21.15.7:0/4271437679 >> v1:172.21.15.7:6790/0 conn(0x7fa234137940 legacy=0x7fa234137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 1 -- 172.21.15.7:0/4271437679 shutdown_connections 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 1 -- 172.21.15.7:0/4271437679 >> 172.21.15.7:0/4271437679 conn(0x7fa2340a9fa0 msgr2=0x7fa2340aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:47.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 1 -- 172.21.15.7:0/4271437679 shutdown_connections 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 1 -- 172.21.15.7:0/4271437679 wait complete. 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.406+0000 7fa23c04e640 10 monclient: build_initial_monmap 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient: monmap: 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:55:47.414 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 1 Processor -- start 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 1 -- start start 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient: init 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient: _reopen_session rank -1 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient: _add_conns ranks=[8,2,5,0,4,6,1,7,3] 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient(hunting): picked mon.noname-e con 0x7fa2340a41b0 addr v1:172.21.15.7:6793/0 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient(hunting): picked mon.noname-b con 0x7fa2340a49a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient(hunting): picked mon.noname-h con 0x7fa234137940 addr v1:172.21.15.81:6791/0 2024-05-08T15:55:47.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa234135900 con 0x7fa2340a49a0 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa234132ad0 con 0x7fa234137940 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2340a1f30 con 0x7fa2340a41b0 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient(hunting): _renew_subs 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa23c04e640 10 monclient(hunting): authenticate will time out at 1355.522435s 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa2395e9640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7fa2340a49a0 0x7fa2341323c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:37120/0 (socket says 172.21.15.7:37120) 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.407+0000 7fa2395e9640 1 -- 172.21.15.7:0/3552751045 learned_addr learned my addr 172.21.15.7:0/3552751045 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880937993 0 0) 0x7fa234135900 con 0x7fa2340a49a0 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): my global_id is 96212 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa218003520 con 0x7fa2340a49a0 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340558586 0 0) 0x7fa2340a1f30 con 0x7fa2340a41b0 2024-05-08T15:55:47.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): my global_id is 144278 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa234135900 con 0x7fa2340a41b0 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3582776414 0 0) 0x7fa234132ad0 con 0x7fa234137940 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): my global_id is 6809 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2340a1f30 con 0x7fa234137940 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3053221643 0 0) 0x7fa234135900 con 0x7fa2340a41b0 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa234132ad0 con 0x7fa2340a41b0 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa230001770 con 0x7fa2340a41b0 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:55:47.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.418 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1204606483 0 0) 0x7fa218003520 con 0x7fa2340a49a0 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa234135900 con 0x7fa2340a49a0 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa224001a90 con 0x7fa2340a49a0 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): got monmap 1 from mon.b (according to old e1) 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.419 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.5 v1:172.21.15.81:6791/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3450047725 0 0) 0x7fa2340a1f30 con 0x7fa234137940 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.81:6791/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7fa218003520 con 0x7fa234137940 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.5 v1:172.21.15.81:6791/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa22c001c80 con 0x7fa234137940 2024-05-08T15:55:47.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): got monmap 1 from mon.h (according to old e1) 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient(hunting): dump: 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 1902316857 0 0) 0x7fa234132ad0 con 0x7fa2340a41b0 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 >> v1:172.21.15.81:6791/0 conn(0x7fa234137940 legacy=0x7fa2341d4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 >> v1:172.21.15.7:6790/0 conn(0x7fa2340a49a0 legacy=0x7fa2341323c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 monclient: found mon.e 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2340a0f60 con 0x7fa2340a41b0 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:55:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa23c04e640 5 monclient: authenticate success, global_id 144278 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa23c04e640 10 monclient: _renew_subs 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa23c04e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7fa234135620 con 0x7fa2340a41b0 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa23c04e640 10 monclient: _renew_subs 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa23c04e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7fa2341346f0 con 0x7fa2340a41b0 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7fa2300011b0 con 0x7fa2340a41b0 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7fa230005100 con 0x7fa2340a41b0 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22bfff640 4 set_mon_vals no callback set 2024-05-08T15:55:47.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.408+0000 7fa22a7fc640 10 monclient: dump: 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:55:47.424 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.409+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7fa230010480 con 0x7fa2340a41b0 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.409+0000 7fa23c04e640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.409+0000 7fa23c04e640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.409+0000 7fa23c04e640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.409+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21c005080 con 0x7fa2340a41b0 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.409+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7fa230046bc0 con 0x7fa2340a41b0 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.410+0000 7fa2395e9640 10 monclient: get_auth_request con 0x7fa21803a3a0 auth_method 0 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.414+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7fa230015ef0 con 0x7fa2340a41b0 2024-05-08T15:55:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.414+0000 7fa22a7fc640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:55:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.414+0000 7fa22a7fc640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:55:47.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.592+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa21c000b90 con 0x7fa21803a3a0 2024-05-08T15:55:47.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.594+0000 7fa22a7fc640 1 -- 172.21.15.7:0/3552751045 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33711 (unknown 2965378022 0 2714972973) 0x7fa21c000b90 con 0x7fa21803a3a0 2024-05-08T15:55:47.590 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:55:47.590 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 >> v1:172.21.15.81:6812/847627118 conn(0x7fa21803a3a0 legacy=0x7fa21803c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 10 monclient: shutdown 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 >> v1:172.21.15.7:6793/0 conn(0x7fa2340a41b0 legacy=0x7fa234131cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 shutdown_connections 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 >> 172.21.15.7:0/3552751045 conn(0x7fa2340a9fa0 msgr2=0x7fa234130990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 shutdown_connections 2024-05-08T15:55:47.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:55:47.596+0000 7fa23c04e640 1 -- 172.21.15.7:0/3552751045 wait complete. 2024-05-08T15:55:47.602 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":579,"stamp":"2024-05-08T15:55:47.208702+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":221716,"kb_used_data":18008,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566009324,"statfs":{"total":579820584960,"available":579593547776,"internally_reserved":0,"allocated":18440192,"data_stored":9126224,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":208525986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002339"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771913+0000","last_change":"2024-05-08T15:47:11.177109+0000","last_active":"2024-05-08T15:54:17.771913+0000","last_peered":"2024-05-08T15:54:17.771913+0000","last_clean":"2024-05-08T15:54:17.771913+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:54:17.771913+0000","last_undegraded":"2024-05-08T15:54:17.771913+0000","last_fullsized":"2024-05-08T15:54:17.771913+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":620,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.776826+0000","last_change":"2024-05-08T15:47:11.176848+0000","last_active":"2024-05-08T15:54:17.776826+0000","last_peered":"2024-05-08T15:54:17.776826+0000","last_clean":"2024-05-08T15:54:17.776826+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:54:17.776826+0000","last_undegraded":"2024-05-08T15:54:17.776826+0000","last_fullsized":"2024-05-08T15:54:17.776826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":627,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:47.023275+0000","last_change":"2024-05-08T15:55:47.023315+0000","last_active":"2024-05-08T15:55:47.023275+0000","last_peered":"2024-05-08T15:55:47.023275+0000","last_clean":"2024-05-08T15:55:47.023275+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:55:47.023275+0000","last_undegraded":"2024-05-08T15:55:47.023275+0000","last_fullsized":"2024-05-08T15:55:47.023275+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:47.023189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:47.023189+0000","last_clean_scrub_stamp":"2024-05-08T15:55:47.023189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:43:32.408932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":616,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.842186+0000","last_change":"2024-05-08T15:47:11.173716+0000","last_active":"2024-05-08T15:54:17.842186+0000","last_peered":"2024-05-08T15:54:17.842186+0000","last_clean":"2024-05-08T15:54:17.842186+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:54:17.842186+0000","last_undegraded":"2024-05-08T15:54:17.842186+0000","last_fullsized":"2024-05-08T15:54:17.842186+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:43:52.603956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":623,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:54:17.771787+0000","last_change":"2024-05-08T15:47:11.173723+0000","last_active":"2024-05-08T15:54:17.771787+0000","last_peered":"2024-05-08T15:54:17.771787+0000","last_clean":"2024-05-08T15:54:17.771787+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:54:17.771787+0000","last_undegraded":"2024-05-08T15:54:17.771787+0000","last_fullsized":"2024-05-08T15:54:17.771787+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:20.853445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":618,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:32.161359+0000","last_change":"2024-05-08T15:47:11.173339+0000","last_active":"2024-05-08T15:55:32.161359+0000","last_peered":"2024-05-08T15:55:32.161359+0000","last_clean":"2024-05-08T15:55:32.161359+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:55:32.161359+0000","last_undegraded":"2024-05-08T15:55:32.161359+0000","last_fullsized":"2024-05-08T15:55:32.161359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":635,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:46.123933+0000","last_change":"2024-05-08T15:55:46.123954+0000","last_active":"2024-05-08T15:55:46.123933+0000","last_peered":"2024-05-08T15:55:46.123933+0000","last_clean":"2024-05-08T15:55:46.123933+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:55:46.123933+0000","last_undegraded":"2024-05-08T15:55:46.123933+0000","last_fullsized":"2024-05-08T15:55:46.123933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:46.123864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:46.123864+0000","last_clean_scrub_stamp":"2024-05-08T15:55:46.123864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:35:07.517845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":619,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:32.161989+0000","last_change":"2024-05-08T15:47:11.174790+0000","last_active":"2024-05-08T15:55:32.161989+0000","last_peered":"2024-05-08T15:55:32.161989+0000","last_clean":"2024-05-08T15:55:32.161989+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:55:32.161989+0000","last_undegraded":"2024-05-08T15:55:32.161989+0000","last_fullsized":"2024-05-08T15:55:32.161989+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:47:10.155258+0000","last_clean_scrub_stamp":"2024-05-08T15:47:10.155258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"323'119","reported_seq":785,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:46.062798+0000","last_change":"2024-05-08T15:55:46.062826+0000","last_active":"2024-05-08T15:55:46.062798+0000","last_peered":"2024-05-08T15:55:46.062798+0000","last_clean":"2024-05-08T15:55:46.062798+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:55:46.062798+0000","last_undegraded":"2024-05-08T15:55:46.062798+0000","last_fullsized":"2024-05-08T15:55:46.062798+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"323'119","last_scrub_stamp":"2024-05-08T15:55:46.062738+0000","last_deep_scrub":"323'119","last_deep_scrub_stamp":"2024-05-08T15:55:46.062738+0000","last_clean_scrub_stamp":"2024-05-08T15:55:46.062738+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:03.808162+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38324,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333516,"statfs":{"total":96636764160,"available":96597520384,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640363,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30052,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341788,"statfs":{"total":96636764160,"available":96605990912,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640364,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640365,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46644,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325196,"statfs":{"total":96636764160,"available":96589000704,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640364,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640363,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30052,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341788,"statfs":{"total":96636764160,"available":96605990912,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:55:47.606 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T15:47:10.155258+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=47, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=55, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T15:55:47.606 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T15:47:10.155258+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=47, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=55, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T15:55:47.606 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T15:47:10.155258+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=47, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=55, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T15:55:47.606 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T15:47:10.155258+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=47, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=55, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T15:55:47.606 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-08T15:47:10.155258+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=47, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=55, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T15:55:47.606 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T15:47:10.155258+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=47, tm_sec=10, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=15, tm_min=55, tm_sec=43, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T15:55:47.606 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T15:56:07.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.728+0000 7f5bcf9ce640 10 monclient: get_monmap_and_config 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.728+0000 7f5bcf9ce640 10 monclient: build_initial_monmap 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.728+0000 7f5bcf9ce640 10 monclient: monmap: 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:56:07.729302+0000 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:56:07.729302+0000 2024-05-08T15:56:07.724 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:56:07.725 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:07.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.728+0000 7f5bcf9ce640 1 Processor -- start 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 1 -- start start 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient: init 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient: _reopen_session rank -1 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient: _add_conns ranks=[4,7,2,5,0,3,6,8,1] 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient(hunting): picked mon.noname-c con 0x7f5bc80a5430 addr v1:172.21.15.7:6791/0 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient(hunting): picked mon.noname-i con 0x7f5bc80a49a0 addr v1:172.21.15.81:6792/0 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient(hunting): picked mon.noname-b con 0x7f5bc813e380 addr v1:172.21.15.7:6790/0 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc80a2990 con 0x7f5bc813e380 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 1 -- --> v1:172.21.15.7:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc80a2c70 con 0x7f5bc80a5430 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 1 -- --> v1:172.21.15.81:6792/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc80a2f50 con 0x7f5bc80a49a0 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient(hunting): _renew_subs 2024-05-08T15:56:07.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcf9ce640 10 monclient(hunting): authenticate will time out at 1375.844481s 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcd76a640 1 --1- >> v1:172.21.15.7:6791/0 conn(0x7f5bc80a5430 0x7f5bc80a5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6791/0 says I am v1:172.21.15.7:40328/0 (socket says 172.21.15.7:40328) 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcd76a640 1 -- 172.21.15.7:0/435941455 learned_addr learned my addr 172.21.15.7:0/435941455 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.729+0000 7f5bcdf6b640 1 --1- 172.21.15.7:0/435941455 >> v1:172.21.15.7:6790/0 conn(0x7f5bc813e380 0x7f5bc8140770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:56400/0 (socket says 172.21.15.7:56400) 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1051269608 0 0) 0x7f5bc80a2990 con 0x7f5bc813e380 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _negotiate 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): my global_id is 96221 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bb4003520 con 0x7f5bc813e380 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 <== mon.4 v1:172.21.15.7:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1479196226 0 0) 0x7f5bc80a2c70 con 0x7f5bc80a5430 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _negotiate 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:07.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): my global_id is 124573 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 --> v1:172.21.15.7:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc80a2990 con 0x7f5bc80a5430 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 <== mon.7 v1:172.21.15.81:6792/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2157225399 0 0) 0x7f5bc80a2f50 con 0x7f5bc80a49a0 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _negotiate 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient(hunting): my global_id is 115423 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 --> v1:172.21.15.81:6792/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc80a2c70 con 0x7f5bc80a49a0 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3215051662 0 0) 0x7f5bb4003520 con 0x7f5bc813e380 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 >> v1:172.21.15.81:6792/0 conn(0x7f5bc80a49a0 legacy=0x7f5bc813cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 >> v1:172.21.15.7:6791/0 conn(0x7f5bc80a5430 legacy=0x7f5bc80a5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: _finish_hunting 0 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 monclient: found mon.noname-b 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:56:07.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc80a4300 con 0x7f5bc813e380 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: _finish_auth 0 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: _check_auth_tickets 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5bc0001a60 con 0x7f5bc813e380 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bcf9ce640 5 monclient: authenticate success, global_id 96221 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bcf9ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: dump: 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.737 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bcf9ce640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5bc0001cc0 con 0x7f5bc813e380 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:56:07.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 1 -- 172.21.15.7:0/435941455 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5bc0004ea0 con 0x7f5bc813e380 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bcf9ce640 10 monclient: get_monmap_and_config success 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bcf9ce640 4 set_mon_vals no callback set 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.730+0000 7f5bbffff640 10 monclient: dump: 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:07.739 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 10 monclient: shutdown 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/435941455 >> v1:172.21.15.7:6790/0 conn(0x7f5bc813e380 legacy=0x7f5bc8140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/435941455 shutdown_connections 2024-05-08T15:56:07.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/435941455 >> 172.21.15.7:0/435941455 conn(0x7f5bc80a9fa0 msgr2=0x7f5bc80aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/435941455 shutdown_connections 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/435941455 wait complete. 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 10 monclient: build_initial_monmap 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.731+0000 7f5bcf9ce640 10 monclient: monmap: 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:56:07.741 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 1 Processor -- start 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 1 -- start start 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient: init 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient: _reopen_session rank -1 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient: _add_conns ranks=[5,8,0,7,6,2,1,4,3] 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient(hunting): picked mon.noname-h con 0x7f5bc80a49a0 addr v1:172.21.15.81:6791/0 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient(hunting): picked mon.noname-e con 0x7f5bc80a5430 addr v1:172.21.15.7:6793/0 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient(hunting): picked mon.noname-a con 0x7f5bc813e380 addr v1:172.21.15.7:6789/0 2024-05-08T15:56:07.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc80a1f30 con 0x7f5bc813e380 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 1 -- --> v1:172.21.15.81:6791/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc8137510 con 0x7f5bc80a49a0 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc81cd280 con 0x7f5bc80a5430 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient(hunting): _renew_subs 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bcf9ce640 10 monclient(hunting): authenticate will time out at 1375.847455s 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bccf69640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f5bc80a5430 0x7f5bc81ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:53712/0 (socket says 172.21.15.7:53712) 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.732+0000 7f5bccf69640 1 -- 172.21.15.7:0/1985178611 learned_addr learned my addr 172.21.15.7:0/1985178611 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2253087722 0 0) 0x7f5bc81cd280 con 0x7f5bc80a5430 2024-05-08T15:56:07.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _negotiate 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): my global_id is 144287 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bac003520 con 0x7f5bc80a5430 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10731630 0 0) 0x7f5bc80a4300 con 0x7f5bc813e380 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _negotiate 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): my global_id is 96021 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc81cd280 con 0x7f5bc813e380 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.5 v1:172.21.15.81:6791/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 334658205 0 0) 0x7f5bc8137510 con 0x7f5bc80a49a0 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _negotiate 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:07.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): my global_id is 6818 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.81:6791/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc80a4300 con 0x7f5bc80a49a0 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1130766727 0 0) 0x7f5bac003520 con 0x7f5bc80a5430 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5bc8137510 con 0x7f5bc80a5430 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5bb8001b00 con 0x7f5bc80a5430 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): got monmap 1 from mon.noname-e (according to old e1) 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): dump: 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:07.745 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.0 v1:172.21.15.7:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3988457079 0 0) 0x7f5bc81cd280 con 0x7f5bc813e380 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f5bac003520 con 0x7f5bc813e380 2024-05-08T15:56:07.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.0 v1:172.21.15.7:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5bc0004ea0 con 0x7f5bc813e380 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): got monmap 1 from mon.a (according to old e1) 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient(hunting): dump: 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:07.747 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 3023654948 0 0) 0x7f5bc8137510 con 0x7f5bc80a5430 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 >> v1:172.21.15.81:6791/0 conn(0x7f5bc80a49a0 legacy=0x7f5bc81cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 >> v1:172.21.15.7:6789/0 conn(0x7f5bc813e380 legacy=0x7f5bc81d49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: _finish_hunting 0 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 monclient: found mon.e 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6793/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc80a0f60 con 0x7f5bc80a5430 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: _finish_auth 0 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: _check_auth_tickets 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:56:07.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bcf9ce640 5 monclient: authenticate success, global_id 144287 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bcf9ce640 10 monclient: _renew_subs 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bcf9ce640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6793/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc8137230 con 0x7f5bc80a5430 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f5bb80038c0 con 0x7f5bc80a5430 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: handle_config config(0 keys) 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f5bb8003e30 con 0x7f5bc80a5430 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: got monmap 1 from mon.e (according to old e1) 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbdffb640 10 monclient: dump: 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:07.749 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bcf9ce640 10 monclient: _renew_subs 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bcf9ce640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bbf7fe640 4 set_mon_vals no callback set 2024-05-08T15:56:07.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.733+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6793/0 -- mon_subscribe({osdmap=0}) -- 0x7f5bc8138770 con 0x7f5bc80a5430 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.734+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f5bb8004090 con 0x7f5bc80a5430 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.734+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f5bb8046d60 con 0x7f5bc80a5430 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.735+0000 7f5bcd76a640 10 monclient: get_auth_request con 0x7f5bac03a240 auth_method 0 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.735+0000 7f5bcf9ce640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.735+0000 7f5bcf9ce640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.735+0000 7f5bcf9ce640 10 monclient: _send_mon_message to mon.e at v1:172.21.15.7:6793/0 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.735+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.7:6793/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc80753d0 con 0x7f5bc80a5430 2024-05-08T15:56:07.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.739+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mon.8 v1:172.21.15.7:6793/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f5bb8016030 con 0x7f5bc80a5430 2024-05-08T15:56:07.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.739+0000 7f5bbdffb640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:56:07.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.739+0000 7f5bbdffb640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:56:07.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.930+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 --> v1:172.21.15.81:6812/847627118 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bc8135490 con 0x7f5bac03a240 2024-05-08T15:56:07.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.932+0000 7f5bbdffb640 1 -- 172.21.15.7:0/1985178611 <== mgr.4108 v1:172.21.15.81:6812/847627118 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+33859 (unknown 2965378022 0 884702342) 0x7f5bc8135490 con 0x7f5bac03a240 2024-05-08T15:56:07.928 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-05-08T15:56:07.928 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-05-08T15:56:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 >> v1:172.21.15.81:6812/847627118 conn(0x7f5bac03a240 legacy=0x7f5bac03c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 10 monclient: shutdown 2024-05-08T15:56:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:56:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 >> v1:172.21.15.7:6793/0 conn(0x7f5bc80a5430 legacy=0x7f5bc81ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 shutdown_connections 2024-05-08T15:56:07.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 >> 172.21.15.7:0/1985178611 conn(0x7f5bc80a9fa0 msgr2=0x7f5bc80a8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:56:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 shutdown_connections 2024-05-08T15:56:07.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:07.934+0000 7f5bcf9ce640 1 -- 172.21.15.7:0/1985178611 wait complete. 2024-05-08T15:56:07.940 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2024-05-08T15:56:07.213090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":566231040,"kb_used":221704,"kb_used_data":18008,"kb_used_omap":9,"kb_used_meta":203638,"kb_avail":566009336,"statfs":{"total":579820584960,"available":579593560064,"internally_reserved":0,"allocated":18440192,"data_stored":9126224,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":9566,"internal_metadata":208525986},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002576"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":627,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:49.051133+0000","last_change":"2024-05-08T15:55:49.051133+0000","last_active":"2024-05-08T15:55:49.051133+0000","last_peered":"2024-05-08T15:55:49.051133+0000","last_clean":"2024-05-08T15:55:49.051133+0000","last_became_active":"2024-05-08T15:47:11.176773+0000","last_became_peered":"2024-05-08T15:47:11.176773+0000","last_unstale":"2024-05-08T15:55:49.051133+0000","last_undegraded":"2024-05-08T15:55:49.051133+0000","last_fullsized":"2024-05-08T15:55:49.051133+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:49.051009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:49.051009+0000","last_clean_scrub_stamp":"2024-05-08T15:55:49.051009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:09:55.246832+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":629,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:47.750677+0000","last_change":"2024-05-08T15:55:47.750677+0000","last_active":"2024-05-08T15:55:47.750677+0000","last_peered":"2024-05-08T15:55:47.750677+0000","last_clean":"2024-05-08T15:55:47.750677+0000","last_became_active":"2024-05-08T15:47:11.176592+0000","last_became_peered":"2024-05-08T15:47:11.176592+0000","last_unstale":"2024-05-08T15:55:47.750677+0000","last_undegraded":"2024-05-08T15:55:47.750677+0000","last_fullsized":"2024-05-08T15:55:47.750677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:47.750541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:47.750541+0000","last_clean_scrub_stamp":"2024-05-08T15:55:47.750541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:43:41.511192+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":627,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:47.023275+0000","last_change":"2024-05-08T15:55:47.023315+0000","last_active":"2024-05-08T15:55:47.023275+0000","last_peered":"2024-05-08T15:55:47.023275+0000","last_clean":"2024-05-08T15:55:47.023275+0000","last_became_active":"2024-05-08T15:47:11.173566+0000","last_became_peered":"2024-05-08T15:47:11.173566+0000","last_unstale":"2024-05-08T15:55:47.023275+0000","last_undegraded":"2024-05-08T15:55:47.023275+0000","last_fullsized":"2024-05-08T15:55:47.023275+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:47.023189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:47.023189+0000","last_clean_scrub_stamp":"2024-05-08T15:55:47.023189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:43:32.408932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":625,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:48.092198+0000","last_change":"2024-05-08T15:55:48.092221+0000","last_active":"2024-05-08T15:55:48.092198+0000","last_peered":"2024-05-08T15:55:48.092198+0000","last_clean":"2024-05-08T15:55:48.092198+0000","last_became_active":"2024-05-08T15:47:11.173236+0000","last_became_peered":"2024-05-08T15:47:11.173236+0000","last_unstale":"2024-05-08T15:55:48.092198+0000","last_undegraded":"2024-05-08T15:55:48.092198+0000","last_fullsized":"2024-05-08T15:55:48.092198+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:48.092098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:48.092098+0000","last_clean_scrub_stamp":"2024-05-08T15:55:48.092098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:28.103045+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":632,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:45.761660+0000","last_change":"2024-05-08T15:55:45.761679+0000","last_active":"2024-05-08T15:55:45.761660+0000","last_peered":"2024-05-08T15:55:45.761660+0000","last_clean":"2024-05-08T15:55:45.761660+0000","last_became_active":"2024-05-08T15:47:11.173430+0000","last_became_peered":"2024-05-08T15:47:11.173430+0000","last_unstale":"2024-05-08T15:55:45.761660+0000","last_undegraded":"2024-05-08T15:55:45.761660+0000","last_fullsized":"2024-05-08T15:55:45.761660+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:45.761576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:45.761576+0000","last_clean_scrub_stamp":"2024-05-08T15:55:45.761576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:04.307322+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":626,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:47.185384+0000","last_change":"2024-05-08T15:55:47.185409+0000","last_active":"2024-05-08T15:55:47.185384+0000","last_peered":"2024-05-08T15:55:47.185384+0000","last_clean":"2024-05-08T15:55:47.185384+0000","last_became_active":"2024-05-08T15:47:11.173039+0000","last_became_peered":"2024-05-08T15:47:11.173039+0000","last_unstale":"2024-05-08T15:55:47.185384+0000","last_undegraded":"2024-05-08T15:55:47.185384+0000","last_fullsized":"2024-05-08T15:55:47.185384+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:47.185294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:47.185294+0000","last_clean_scrub_stamp":"2024-05-08T15:55:47.185294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:37:02.543819+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":635,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:46.123933+0000","last_change":"2024-05-08T15:55:46.123954+0000","last_active":"2024-05-08T15:55:46.123933+0000","last_peered":"2024-05-08T15:55:46.123933+0000","last_clean":"2024-05-08T15:55:46.123933+0000","last_became_active":"2024-05-08T15:47:11.176511+0000","last_became_peered":"2024-05-08T15:47:11.176511+0000","last_unstale":"2024-05-08T15:55:46.123933+0000","last_undegraded":"2024-05-08T15:55:46.123933+0000","last_fullsized":"2024-05-08T15:55:46.123933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:46.123864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:46.123864+0000","last_clean_scrub_stamp":"2024-05-08T15:55:46.123864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:35:07.517845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":627,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:48.065515+0000","last_change":"2024-05-08T15:55:48.065538+0000","last_active":"2024-05-08T15:55:48.065515+0000","last_peered":"2024-05-08T15:55:48.065515+0000","last_clean":"2024-05-08T15:55:48.065515+0000","last_became_active":"2024-05-08T15:47:11.174384+0000","last_became_peered":"2024-05-08T15:47:11.174384+0000","last_unstale":"2024-05-08T15:55:48.065515+0000","last_undegraded":"2024-05-08T15:55:48.065515+0000","last_fullsized":"2024-05-08T15:55:48.065515+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T15:55:48.065428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T15:55:48.065428+0000","last_clean_scrub_stamp":"2024-05-08T15:55:48.065428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:12:37.137602+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"323'119","reported_seq":785,"reported_epoch":323,"state":"active+clean","last_fresh":"2024-05-08T15:55:46.062798+0000","last_change":"2024-05-08T15:55:46.062826+0000","last_active":"2024-05-08T15:55:46.062798+0000","last_peered":"2024-05-08T15:55:46.062798+0000","last_clean":"2024-05-08T15:55:46.062798+0000","last_became_active":"2024-05-08T15:47:04.947972+0000","last_became_peered":"2024-05-08T15:47:04.947972+0000","last_unstale":"2024-05-08T15:55:46.062798+0000","last_undegraded":"2024-05-08T15:55:46.062798+0000","last_fullsized":"2024-05-08T15:55:46.062798+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"323'119","last_scrub_stamp":"2024-05-08T15:55:46.062738+0000","last_deep_scrub":"323'119","last_deep_scrub_stamp":"2024-05-08T15:55:46.062738+0000","last_clean_scrub_stamp":"2024-05-08T15:55:46.062738+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:03.808162+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11038,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":13,"seq":55834574960,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38320,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333520,"statfs":{"total":96636764160,"available":96597524480,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":11,"seq":47244640367,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30052,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341788,"statfs":{"total":96636764160,"available":96605990912,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":11,"seq":47244640368,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46640,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325200,"statfs":{"total":96636764160,"available":96589004800,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":11,"seq":47244640369,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46640,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94325200,"statfs":{"total":96636764160,"available":96589004800,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":44693960},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640368,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30000,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341840,"statfs":{"total":96636764160,"available":96606044160,"internally_reserved":0,"allocated":3055616,"data_stored":1344728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640367,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30052,"kb_used_data":3036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94341788,"statfs":{"total":96636764160,"available":96605990912,"internally_reserved":0,"allocated":3108864,"data_stored":1873656,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5519,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T15:56:07.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T15:56:08.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.048+0000 7f1f91dd4640 10 monclient: get_monmap_and_config 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.048+0000 7f1f91dd4640 10 monclient: build_initial_monmap 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.048+0000 7f1f91dd4640 10 monclient: monmap: 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:56:08.049096+0000 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:56:08.049096+0000 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:56:08.044 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:56:08.045 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:56:08.045 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:56:08.045 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:56:08.045 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:56:08.045 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:56:08.045 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.048+0000 7f1f91dd4640 1 Processor -- start 2024-05-08T15:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.048+0000 7f1f91dd4640 1 -- start start 2024-05-08T15:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.048+0000 7f1f91dd4640 10 monclient: init 2024-05-08T15:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 10 monclient: _reopen_session rank -1 2024-05-08T15:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 10 monclient: _add_conns ranks=[2,3,0,1,5,8,7,6,4] 2024-05-08T15:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 10 monclient(hunting): picked mon.noname-b con 0x7f1f8c0a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:56:08.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 10 monclient(hunting): picked mon.noname-g con 0x7f1f8c0a5780 addr v1:172.21.15.81:6790/0 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 10 monclient(hunting): picked mon.noname-a con 0x7f1f8c13e310 addr v1:172.21.15.7:6789/0 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 1 -- --> v1:172.21.15.7:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c0a1660 con 0x7f1f8c13e310 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c0a28f0 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 1 -- --> v1:172.21.15.81:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c0a2bd0 con 0x7f1f8c0a5780 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 10 monclient(hunting): _renew_subs 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f91dd4640 10 monclient(hunting): authenticate will time out at 1376.164025s 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8b7fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f1f8c0a53a0 0x7f1f8c0a49a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:56406/0 (socket says 172.21.15.7:56406) 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8bfff640 1 --1- >> v1:172.21.15.7:6789/0 conn(0x7f1f8c13e310 0x7f1f8c140700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.7:6789/0 says I am v1:172.21.15.7:35032/0 (socket says 172.21.15.7:35032) 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8b7fe640 1 -- 172.21.15.7:0/1832312873 learned_addr learned my addr 172.21.15.7:0/1832312873 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463351684 0 0) 0x7f1f8c0a28f0 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:08.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): my global_id is 96230 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f74003520 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 <== mon.0 v1:172.21.15.7:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2454116455 0 0) 0x7f1f8c0a1660 con 0x7f1f8c13e310 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 10 monclient(hunting): my global_id is 96030 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.049+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 --> v1:172.21.15.7:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f8c0a28f0 con 0x7f1f8c13e310 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 <== mon.3 v1:172.21.15.81:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660430204 0 0) 0x7f1f8c0a2bd0 con 0x7f1f8c0a5780 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient(hunting): _negotiate 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient(hunting): my global_id is 77088 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 --> v1:172.21.15.81:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f8c0a1660 con 0x7f1f8c0a5780 2024-05-08T15:56:08.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2664015749 0 0) 0x7f1f74003520 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 >> v1:172.21.15.81:6790/0 conn(0x7f1f8c0a5780 legacy=0x7f1f8c13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 >> v1:172.21.15.7:6789/0 conn(0x7f1f8c13e310 legacy=0x7f1f8c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: _finish_hunting 0 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 monclient: found mon.noname-b 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: _send_mon_message to mon.noname-b at v1:172.21.15.7:6790/0 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c0a6730 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: _finish_auth 0 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: _check_auth_tickets 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:56:08.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 5 monclient: authenticate success, global_id 96230 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1f7c0017b0 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: dump: 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:08.056 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 20 monclient: get_monmap_and_config waiting for monmap|config 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 <== mon.2 v1:172.21.15.7:6790/0 4 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1f7c001a10 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: handle_config config(0 keys) 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 1 -- 172.21.15.7:0/1832312873 <== mon.2 v1:172.21.15.7:6790/0 5 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1f7c004be0 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f8a7fc640 10 monclient: dump: 2024-05-08T15:56:08.057 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:08.058 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 10 monclient: get_monmap_and_config success 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 4 set_mon_vals no callback set 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 10 monclient: shutdown 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 1 -- 172.21.15.7:0/1832312873 >> v1:172.21.15.7:6790/0 conn(0x7f1f8c0a53a0 legacy=0x7f1f8c0a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 1 -- 172.21.15.7:0/1832312873 shutdown_connections 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 1 -- 172.21.15.7:0/1832312873 >> 172.21.15.7:0/1832312873 conn(0x7f1f8c0a9fa0 msgr2=0x7f1f8c0aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 1 -- 172.21.15.7:0/1832312873 shutdown_connections 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.050+0000 7f1f91dd4640 1 -- 172.21.15.7:0/1832312873 wait complete. 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.051+0000 7f1f91dd4640 10 monclient: build_initial_monmap 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.051+0000 7f1f91dd4640 10 monclient: monmap: 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 0 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 00000000-0000-0000-0000-000000000000 2024-05-08T15:56:08.059 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 0.000000 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:created 0.000000 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 0 (unknown) 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 1 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.noname-a 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.noname-f 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.noname-b 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.noname-g 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.noname-c 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.noname-h 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.noname-d 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.noname-i 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.noname-e 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.051+0000 7f1f91dd4640 1 Processor -- start 2024-05-08T15:56:08.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.051+0000 7f1f91dd4640 1 -- start start 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.051+0000 7f1f91dd4640 10 monclient: init 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 10 monclient: _reopen_session rank -1 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 10 monclient: _add_conns ranks=[2,1,8,0,4,5,6,7,3] 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 10 monclient(hunting): picked mon.noname-b con 0x7f1f8c0a53a0 addr v1:172.21.15.7:6790/0 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 10 monclient(hunting): picked mon.noname-f con 0x7f1f8c0a5780 addr v1:172.21.15.81:6789/0 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 10 monclient(hunting): picked mon.noname-e con 0x7f1f8c13e310 addr v1:172.21.15.7:6793/0 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c137a30 con 0x7f1f8c0a5780 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 1 -- --> v1:172.21.15.7:6790/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c0a1e00 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 1 -- --> v1:172.21.15.7:6793/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f8c0a43c0 con 0x7f1f8c13e310 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 10 monclient(hunting): _renew_subs 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f91dd4640 10 monclient(hunting): authenticate will time out at 1376.166942s 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f8bfff640 1 --1- >> v1:172.21.15.7:6793/0 conn(0x7f1f8c13e310 0x7f1f8c1d4d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6793/0 says I am v1:172.21.15.7:53714/0 (socket says 172.21.15.7:53714) 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f8bfff640 1 -- 172.21.15.7:0/4171278274 learned_addr learned my addr 172.21.15.7:0/4171278274 (peer_addr_for_me v1:172.21.15.7:0/0) 2024-05-08T15:56:08.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f8b7fe640 1 --1- >> v1:172.21.15.7:6790/0 conn(0x7f1f8c0a53a0 0x7f1f8c133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.7:6790/0 says I am v1:172.21.15.7:56410/0 (socket says 172.21.15.7:56410) 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 479592034 0 0) 0x7f1f8c0a1e00 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _negotiate 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): my global_id is 96239 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6790/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f78003520 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.8 v1:172.21.15.7:6793/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 929885438 0 0) 0x7f1f8c0a43c0 con 0x7f1f8c13e310 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _negotiate 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): my global_id is 144296 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6793/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f8c0a1e00 con 0x7f1f8c13e310 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.1 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198391469 0 0) 0x7f1f8c137a30 con 0x7f1f8c0a5780 2024-05-08T15:56:08.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _negotiate 2024-05-08T15:56:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _init_auth method 2 2024-05-08T15:56:08.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): _init_auth creating new auth 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): my global_id is 6634 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f8c0a43c0 con 0x7f1f8c0a5780 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 3989511490 0 0) 0x7f1f78003520 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6790/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1f8c137a30 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1f7c004b20 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): got monmap 1 from mon.noname-b (according to old e1) 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.052+0000 7f1f73fff640 10 monclient(hunting): dump: 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.064 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient(hunting): _finish_auth 0 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.8 v1:172.21.15.7:6793/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1534215108 0 0) 0x7f1f8c0a1e00 con 0x7f1f8c13e310 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6793/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1f78003520 con 0x7f1f8c13e310 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.1 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 764+0+0 (unknown 1448092491 0 0) 0x7f1f8c0a43c0 con 0x7f1f8c0a5780 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.81:6789/0 -- auth(proto 2 165 bytes epoch 0) -- 0x7f1f8c0a1e00 con 0x7f1f8c0a5780 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.8 v1:172.21.15.7:6793/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1f84001c80 con 0x7f1f8c13e310 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient(hunting): got monmap 1 from mon.e (according to old e1) 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient(hunting): dump: 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:08.066 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:08.067 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.1 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1f800019d0 con 0x7f1f8c0a5780 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient(hunting): handle_monmap mon_map magic: 0 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient(hunting): got monmap 1 from mon.f (according to old e1) 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient(hunting): dump: 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:08.068 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 580+0+0 (unknown 582213672 0 0) 0x7f1f8c137a30 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 >> v1:172.21.15.7:6793/0 conn(0x7f1f8c13e310 legacy=0x7f1f8c1d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 >> v1:172.21.15.81:6789/0 conn(0x7f1f8c0a5780 legacy=0x7f1f8c1344a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: _finish_hunting 0 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 monclient: found mon.b 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 20 monclient: _un_backoff reopen_interval_multipler now 1 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6790/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f8c135a20 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: _finish_auth 0 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: _check_auth_tickets 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 20 monclient: _check_auth_rotating not needed by client.admin 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f91dd4640 5 monclient: authenticate success, global_id 96239 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f91dd4640 10 monclient: _renew_subs 2024-05-08T15:56:08.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f91dd4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6790/0 -- mon_subscribe({mgrmap=0+}) -- 0x7f1f8c134bb0 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f91dd4640 10 monclient: _renew_subs 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f91dd4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6790/0 -- mon_subscribe({osdmap=0}) -- 0x7f1f8c137d10 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 5 ==== config(0 keys) ==== 4+0+0 (unknown 0 0 0) 0x7f1f7c001a10 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: handle_config config(0 keys) 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 6 ==== mon_map magic: 0 ==== 722+0+0 (unknown 74490335 0 0) 0x7f1f7c001790 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: handle_monmap mon_map magic: 0 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f89ffb640 4 set_mon_vals no callback set 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: got monmap 1 from mon.b (according to old e1) 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.053+0000 7f1f73fff640 10 monclient: dump: 2024-05-08T15:56:08.071 INFO:teuthology.orchestra.run.smithi007.stderr:epoch 1 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:fsid 402a637b-159c-4d0b-9f9c-7ff72b154b33 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:last_changed 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:created 2024-05-08T15:46:23.145395+0000 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:min_mon_release 19 (squid) 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:election_strategy: 3 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:0: v1:172.21.15.7:6789/0 mon.a 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:1: v1:172.21.15.81:6789/0 mon.f 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:2: v1:172.21.15.7:6790/0 mon.b 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:3: v1:172.21.15.81:6790/0 mon.g 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:4: v1:172.21.15.7:6791/0 mon.c 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:5: v1:172.21.15.81:6791/0 mon.h 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:6: v1:172.21.15.7:6792/0 mon.d 2024-05-08T15:56:08.072 INFO:teuthology.orchestra.run.smithi007.stderr:7: v1:172.21.15.81:6792/0 mon.i 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:8: v1:172.21.15.7:6793/0 mon.e 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.054+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 7 ==== mgrmap(e 5) ==== 45528+0+0 (unknown 680304075 0 0) 0x7f1f7c010410 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.054+0000 7f1f91dd4640 10 start_mon_command cmd=[{"prefix": "get_command_descriptions"}] 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.054+0000 7f1f91dd4640 10 monclient: _send_command 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.054+0000 7f1f91dd4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.054+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f6c005080 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.054+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 8 ==== osd_map(323..323 src has 62..323) ==== 3506+0+0 (unknown 617131251 0 0) 0x7f1f7c046b70 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.054+0000 7f1f8affd640 10 monclient: get_auth_request con 0x7f1f7803a3f0 auth_method 0 2024-05-08T15:56:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.058+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (unknown 1092875540 0 2154935188) 0x7f1f7c015e90 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.058+0000 7f1f73fff640 10 monclient: handle_mon_command_ack 1 [{"prefix": "get_command_descriptions"}] 2024-05-08T15:56:08.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.058+0000 7f1f73fff640 10 monclient: _finish_command 1 = system:0 2024-05-08T15:56:08.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.243+0000 7f1f91dd4640 10 start_mon_command cmd=[{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-05-08T15:56:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.243+0000 7f1f91dd4640 10 monclient: _send_command 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-05-08T15:56:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.243+0000 7f1f91dd4640 10 monclient: _send_mon_message to mon.b at v1:172.21.15.7:6790/0 2024-05-08T15:56:08.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.243+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 --> v1:172.21.15.7:6790/0 -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f1f6c005370 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.247+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 10 ==== config(1 keys) ==== 35+0+0 (unknown 2555425225 0 0) 0x7f1f7c015790 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.247+0000 7f1f73fff640 10 monclient: handle_config config(1 keys) 2024-05-08T15:56:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.247+0000 7f1f73fff640 1 -- 172.21.15.7:0/4171278274 <== mon.2 v1:172.21.15.7:6790/0 11 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v3)=0 v3) ==== 123+0+0 (unknown 3022875181 0 0) 0x7f1f7c013af0 con 0x7f1f8c0a53a0 2024-05-08T15:56:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.247+0000 7f1f73fff640 10 monclient: handle_mon_command_ack 2 [{"prefix": "config set", "who": "global", "name": "mon_health_to_clog", "value": "false"}] 2024-05-08T15:56:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.247+0000 7f1f73fff640 10 monclient: _finish_command 2 = system:0 2024-05-08T15:56:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.247+0000 7f1f897fa640 4 set_mon_vals no callback set 2024-05-08T15:56:08.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.247+0000 7f1f897fa640 10 set_mon_vals mon_health_to_clog = false 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 >> v1:172.21.15.81:6812/847627118 conn(0x7f1f7803a3f0 legacy=0x7f1f7803c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 10 monclient: shutdown 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 20 monclient: shutdown discarding 0 pending message(s) 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 >> v1:172.21.15.7:6790/0 conn(0x7f1f8c0a53a0 legacy=0x7f1f8c133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 shutdown_connections 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 >> 172.21.15.7:0/4171278274 conn(0x7f1f8c0a9fa0 msgr2=0x7f1f8c0a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 shutdown_connections 2024-05-08T15:56:08.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-05-08T15:56:08.250+0000 7f1f91dd4640 1 -- 172.21.15.7:0/4171278274 wait complete. 2024-05-08T15:56:08.257 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T15:56:08.257 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T15:56:08.257 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T15:56:08.257 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.300 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T15:56:08.300 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T15:56:08.300 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.351 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T15:56:08.351 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T15:56:08.352 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.395 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T15:56:08.395 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T15:56:08.395 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.433 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T15:56:08.434 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T15:56:08.434 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.502 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T15:56:08.503 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T15:56:08.503 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.575 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T15:56:08.575 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T15:56:08.575 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T15:56:08.575 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.645 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T15:56:08.645 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T15:56:08.646 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T15:56:08.646 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.658 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T15:56:08.658 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T15:56:08.658 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.711 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T15:56:08.711 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T15:56:08.711 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.763 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T15:56:08.763 DEBUG:tasks.ceph.mon.d:waiting for process to exit 2024-05-08T15:56:08.763 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.815 INFO:tasks.ceph.mon.d:Stopped 2024-05-08T15:56:08.815 DEBUG:tasks.ceph.mon.e:waiting for process to exit 2024-05-08T15:56:08.815 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.865 INFO:tasks.ceph.mon.e:Stopped 2024-05-08T15:56:08.865 DEBUG:tasks.ceph.mon.f:waiting for process to exit 2024-05-08T15:56:08.865 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.881 INFO:tasks.ceph.mon.f:Stopped 2024-05-08T15:56:08.882 DEBUG:tasks.ceph.mon.g:waiting for process to exit 2024-05-08T15:56:08.882 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.936 INFO:tasks.ceph.mon.g:Stopped 2024-05-08T15:56:08.937 DEBUG:tasks.ceph.mon.h:waiting for process to exit 2024-05-08T15:56:08.937 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:08.991 INFO:tasks.ceph.mon.h:Stopped 2024-05-08T15:56:08.991 DEBUG:tasks.ceph.mon.i:waiting for process to exit 2024-05-08T15:56:08.991 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T15:56:09.044 INFO:tasks.ceph.mon.i:Stopped 2024-05-08T15:56:09.044 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T15:56:09.044 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v ScrubResult | egrep -v 'scrub mismatch' | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'slow request' | egrep -v 'overall HEALTH_' | head -n 1 2024-05-08T15:56:09.080 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:56:09.081 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T15:56:09.306 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:56:09.306 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T15:56:09.400 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:56:09.400 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T15:56:09.476 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:56:09.476 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T15:56:09.640 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:56:09.640 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T15:56:09.709 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:56:09.710 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T15:56:09.784 INFO:tasks.ceph:Archiving mon data... 2024-05-08T15:56:09.786 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.a.tgz 2024-05-08T15:56:09.786 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-08T15:56:09.803 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.lV6ktkzaIm 2024-05-08T15:56:09.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.lV6ktkzaIm 2024-05-08T15:56:09.919 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.lV6ktkzaIm 2024-05-08T15:56:10.005 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.lV6ktkzaIm is 248KB 2024-05-08T15:56:10.101 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.lV6ktkzaIm 2024-05-08T15:56:10.117 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.b.tgz 2024-05-08T15:56:10.117 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-08T15:56:10.173 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.7dqc6vdXxH 2024-05-08T15:56:10.173 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7dqc6vdXxH 2024-05-08T15:56:10.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.7dqc6vdXxH 2024-05-08T15:56:10.362 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.7dqc6vdXxH is 249KB 2024-05-08T15:56:10.455 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.7dqc6vdXxH 2024-05-08T15:56:10.470 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.c.tgz 2024-05-08T15:56:10.471 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-08T15:56:10.526 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.JSXks85tqF 2024-05-08T15:56:10.527 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.JSXks85tqF 2024-05-08T15:56:10.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.JSXks85tqF 2024-05-08T15:56:10.715 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.JSXks85tqF is 241KB 2024-05-08T15:56:10.808 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.JSXks85tqF 2024-05-08T15:56:10.824 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-d to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.d.tgz 2024-05-08T15:56:10.824 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-08T15:56:10.880 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.an5xwXNSef 2024-05-08T15:56:10.881 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-d -- . > /tmp/tmp.an5xwXNSef 2024-05-08T15:56:10.983 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.an5xwXNSef 2024-05-08T15:56:11.068 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.an5xwXNSef is 243KB 2024-05-08T15:56:11.161 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.an5xwXNSef 2024-05-08T15:56:11.177 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-e to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.e.tgz 2024-05-08T15:56:11.177 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-05-08T15:56:11.232 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.jHeR9FTVE4 2024-05-08T15:56:11.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-e -- . > /tmp/tmp.jHeR9FTVE4 2024-05-08T15:56:11.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.jHeR9FTVE4 2024-05-08T15:56:11.419 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.jHeR9FTVE4 is 240KB 2024-05-08T15:56:11.513 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.jHeR9FTVE4 2024-05-08T15:56:11.529 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-f to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.f.tgz 2024-05-08T15:56:11.529 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-05-08T15:56:11.548 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.HP3nECAnEO 2024-05-08T15:56:11.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-f -- . > /tmp/tmp.HP3nECAnEO 2024-05-08T15:56:11.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.HP3nECAnEO 2024-05-08T15:56:11.783 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.HP3nECAnEO is 253KB 2024-05-08T15:56:11.878 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.HP3nECAnEO 2024-05-08T15:56:11.894 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-g to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.g.tgz 2024-05-08T15:56:11.894 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-05-08T15:56:11.949 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.7v6ZXytEuB 2024-05-08T15:56:11.949 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-g -- . > /tmp/tmp.7v6ZXytEuB 2024-05-08T15:56:12.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.7v6ZXytEuB 2024-05-08T15:56:12.135 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.7v6ZXytEuB is 253KB 2024-05-08T15:56:12.229 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.7v6ZXytEuB 2024-05-08T15:56:12.244 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-h to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.h.tgz 2024-05-08T15:56:12.245 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-05-08T15:56:12.299 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.fhwpOC1nxU 2024-05-08T15:56:12.299 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-h -- . > /tmp/tmp.fhwpOC1nxU 2024-05-08T15:56:12.401 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.fhwpOC1nxU 2024-05-08T15:56:12.484 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.fhwpOC1nxU is 253KB 2024-05-08T15:56:12.550 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.fhwpOC1nxU 2024-05-08T15:56:12.565 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-i to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/data/mon.i.tgz 2024-05-08T15:56:12.565 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-05-08T15:56:12.619 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.q4ZYPTCPq4 2024-05-08T15:56:12.620 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-i -- . > /tmp/tmp.q4ZYPTCPq4 2024-05-08T15:56:12.720 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.q4ZYPTCPq4 2024-05-08T15:56:12.803 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.q4ZYPTCPq4 is 246KB 2024-05-08T15:56:12.897 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.q4ZYPTCPq4 2024-05-08T15:56:12.912 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T15:56:12.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T15:56:12.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T15:56:13.003 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T15:56:13.031 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T15:56:13.031 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T15:56:13.032 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/remote/smithi007/crash 2024-05-08T15:56:13.033 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T15:56:13.072 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/crash to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/remote/smithi081/crash 2024-05-08T15:56:13.073 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T15:56:13.097 INFO:tasks.ceph:Compressing logs... 2024-05-08T15:56:13.097 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T15:56:13.117 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T15:56:13.144 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34404.log 2024-05-08T15:56:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41299.log 2024-05-08T15:56:13.147 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39168.log 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36544.log 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34404.log.gz 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39168.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41299.log.gz 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38497.log 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38708.log 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39168.log.gz 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38497.log: gzip 89.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.36544.log.gz --verbose 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.40230.log 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38497.log.gz 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40230.log: gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.38708.log.gz 2024-05-08T15:56:13.148 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36214.log 2024-05-08T15:56:13.370 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41089.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38666.log 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40711.log 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38666.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36214.log.gz 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37384.log 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40711.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.38666.log.gz 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41593.log 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35273.log 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40711.log.gz 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41593.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.37384.log.gz -5 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37510.log 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-05-08T15:56:13.371 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41593.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34223.log 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35273.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40753.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37510.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41257.log 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38161.log 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41467.log 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34223.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35485.log 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41257.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38329.log 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38161.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35485.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41467.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38329.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35485.log.gz 2024-05-08T15:56:13.372 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35569.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38329.log.gz 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36880.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40020.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36880.log: gzip 89.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.35569.log.gz -- 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-mon.d.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39420.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36880.log.gz 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40062.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40020.log.gz 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34655.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40062.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39420.log.gz 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34655.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40062.log.gz 2024-05-08T15:56:13.373 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34653.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34655.log.gz 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34970.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.34653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34659.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34970.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34653.log.gz 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.e.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40459.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34659.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34970.log.gz 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41341.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34659.log.gz 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33929.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41341.log: gzip -5 --verbose -- 89.0% /var/log/ceph/ceph.audit.log 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40459.log.gz 2024-05-08T15:56:13.374 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.33929.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41341.log.gz 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36838.log 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33929.log.gz 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35653.log 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35399.log 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35653.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37698.log 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36838.log.gz 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35653.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41551.log 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35399.log: /var/log/ceph/ceph-client.admin.37698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35737.log 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35399.log.gz 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41551.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37698.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38750.log 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:13.375 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41551.log.gz 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37342.log 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35737.log: gzip/var/log/ceph/ceph-client.admin.38750.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34911.log 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr: 96.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37342.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38750.log.gz 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35737.log.gz 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37342.log.gz 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40795.log 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40669.log 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40146.log 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34911.log.gz 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40188.log 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40795.log.gz 2024-05-08T15:56:13.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.40146.log: -- 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40669.log.gz 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.39978.log 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40188.log: gzip -5 89.6% --verbose -- /var/log/ceph/ceph-client.admin.40627.log 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40146.log.gz 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40188.log.gz 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37824.log 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40627.log: gzip -5 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39978.log.gz 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36418.log 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37824.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40627.log.gz 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36628.log 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37824.log.gz 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37090.log 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36628.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36418.log.gz 2024-05-08T15:56:13.377 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40921.log 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36628.log.gz 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34831.log 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37090.log.gz 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41383.log 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34831.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40921.log.gz 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38834.log 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4%/var/log/ceph/ceph-client.admin.41383.log: -- replaced with /var/log/ceph/ceph-client.admin.34831.log.gz 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39504.log 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41383.log.gz 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34407.log 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39504.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37006.log 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38834.log.gz 2024-05-08T15:56:13.378 INFO:teuthology.orchestra.run.smithi007.stderr: 94.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34407.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39504.log.gz 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35611.log 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34407.log.gz 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37006.log: /var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37950.log 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37006.log.gz 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38076.log 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37950.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35611.log.gz 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37258.log 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37950.log.gz 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38287.log 2024-05-08T15:56:13.379 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37258.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38076.log.gz 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37992.log 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37258.log.gz 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38287.log: /var/log/ceph/ceph-client.admin.37992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35905.log 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41047.log 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37992.log.gz 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3%/var/log/ceph/ceph-client.admin.35905.log: -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38624.log 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr: 89.6%/var/log/ceph/ceph-client.admin.41047.log: -- replaced with /var/log/ceph/ceph-client.admin.35905.log.gz 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38413.log 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41047.log.gz 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2024-05-08T15:56:13.380 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38624.log: /var/log/ceph/ceph-client.admin.38413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36922.log 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36754.log: 88.6% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38413.log.gz 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38624.log.gz 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35357.log 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39630.log 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35357.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36922.log.gz 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38582.log 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38455.log 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35357.log.gz 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39630.log: /var/log/ceph/ceph-client.admin.38582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38119.log 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38455.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38582.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39630.log.gz 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:13.381 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34398.log 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38455.log.gz 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41425.log 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38119.log: gzip/var/log/ceph/ceph-client.admin.34398.log: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38119.log.gz 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34398.log.gz 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2%/var/log/ceph/ceph-mon.c.log: -- replaced with /var/log/ceph/ceph-client.admin.41425.log.gz 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37174.log 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36376.log 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr: 89.4%/var/log/ceph/ceph-client.admin.37174.log: -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37174.log.gz 2024-05-08T15:56:13.382 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35231.log 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35231.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36376.log.gz 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31075.log 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35231.log.gz 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41215.log 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31075.log: /var/log/ceph/ceph-client.admin.40501.log: 88.3% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31075.log.gz 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41633.log 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41215.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41215.log.gz 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40374.log 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41633.log.gz 2024-05-08T15:56:13.383 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39924.log 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40374.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40374.log.gz 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36031.log 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39924.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39924.log.gz 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38371.log 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36031.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36031.log.gz 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40585.log 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38371.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38371.log.gz 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35063.log 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40585.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40585.log.gz 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36115.log 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35063.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35063.log.gz 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34650.log 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36115.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36115.log.gz 2024-05-08T15:56:13.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40417.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34650.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34650.log.gz 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36460.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40417.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40417.log.gz 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34402.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36460.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36460.log.gz 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36964.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34402.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34402.log.gz 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36586.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36964.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36964.log.gz 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36586.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36586.log.gz 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34041.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39125.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39294.log 2024-05-08T15:56:13.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34041.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34041.log.gz 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37617.log 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39294.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39294.log.gz 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39882.log 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37617.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37617.log.gz 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35527.log 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39882.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39882.log.gz 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40963.log 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35527.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35527.log.gz 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39588.log 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40963.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40963.log.gz 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34406.log 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39588.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39588.log.gz 2024-05-08T15:56:13.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26431.log 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34406.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34406.log.gz 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39840.log 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.26431.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26431.log.gz 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39462.log 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39840.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39840.log.gz 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37782.log 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39462.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39462.log.gz 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34971.log 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37782.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37782.log.gz 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34971.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34971.log.gz 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38792.log 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39041.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-05-08T15:56:13.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39083.log 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38792.log.gz 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40104.log 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39083.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39083.log.gz 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34657.log 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40104.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40104.log.gz 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34657.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34657.log.gz 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36712.log 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34139.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38876.log 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36712.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36712.log.gz 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34658.log 2024-05-08T15:56:13.388 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38876.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38876.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36502.log 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34658.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34658.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39756.log 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36502.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36502.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38245.log 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39756.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39756.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31499.log 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38245.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38245.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37866.log 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31499.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31499.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35779.log 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37866.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37866.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39252.log 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35779.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35779.log.gz 2024-05-08T15:56:13.389 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37048.log 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39252.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39252.log.gz 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35189.log 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37048.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37048.log.gz 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39798.log 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35189.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35189.log.gz 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34097.log 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39798.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39798.log.gz 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37300.log 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34097.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34097.log.gz 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37300.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37300.log.gz 2024-05-08T15:56:13.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39336.log 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36073.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36796.log 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39336.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39336.log.gz 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37132.log 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36796.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36796.log.gz 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41005.log 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37132.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37132.log.gz 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34403.log 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41005.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41005.log.gz 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37468.log 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34403.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34403.log.gz 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37216.log 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37468.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37468.log.gz 2024-05-08T15:56:13.391 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40879.log 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37216.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37216.log.gz 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40879.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40879.log.gz 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35947.log 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37426.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35147.log 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35947.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35947.log.gz 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34864.log 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35147.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35147.log.gz 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37740.log 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34864.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34864.log.gz 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39210.log 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37740.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37740.log.gz 2024-05-08T15:56:13.392 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34811.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39210.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39210.log.gz 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39378.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34811.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34811.log.gz 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34265.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39378.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39378.log.gz 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39672.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34265.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34265.log.gz 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39546.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39672.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39672.log.gz 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37552.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39546.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39546.log.gz 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41509.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.37552.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37552.log.gz 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38034.log 2024-05-08T15:56:13.393 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41509.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41509.log.gz 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35863.log 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38034.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38034.log.gz 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35443.log 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35863.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35863.log.gz 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41173.log 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35443.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35443.log.gz 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41173.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41173.log.gz 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35821.log 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36670.log 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35821.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35821.log.gz 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31425.log 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36670.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36670.log.gz 2024-05-08T15:56:13.394 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34181.log 2024-05-08T15:56:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.31425.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31425.log.gz 2024-05-08T15:56:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40837.log 2024-05-08T15:56:13.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.34181.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34181.log.gz 2024-05-08T15:56:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35989.log 2024-05-08T15:56:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40837.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40837.log.gz 2024-05-08T15:56:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38203.log 2024-05-08T15:56:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35989.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35989.log.gz 2024-05-08T15:56:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40272.log 2024-05-08T15:56:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38203.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38203.log.gz 2024-05-08T15:56:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41131.log 2024-05-08T15:56:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40272.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40272.log.gz 2024-05-08T15:56:13.416 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36800.log 2024-05-08T15:56:13.416 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35835.log 2024-05-08T15:56:13.416 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36228.log 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34880.log 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36800.log.gz 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35203.log 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34880.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35835.log.gz 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38692.log 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36549.log 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34880.log.gz 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36228.log.gz 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37900.log 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35203.log.gz 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38692.log.gz/var/log/ceph/ceph-client.admin.36549.log: 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36763.log 2024-05-08T15:56:13.417 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35238.log 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35309.log 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36549.log.gz 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37900.log.gz 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35202.log 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38043.log 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36763.log.gz 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36336.log 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35238.log.gz 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37085.log 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35309.log.gz 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36336.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38043.log.gz 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35202.log.gz 2024-05-08T15:56:13.418 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35096.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36336.log.gz 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34558.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34595.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37085.log.gz 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34344.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35096.log.gz 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38256.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39314.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34558.log.gz 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34595.log.gz 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34808.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34665.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34344.log.gz 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34808.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39314.log.gz 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38221.log 2024-05-08T15:56:13.419 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38256.log.gz 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36906.log 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31313.log 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34808.log.gz 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36906.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38221.log.gz 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35693.log 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34665.log.gz 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.31313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35477.log 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34237.log 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31313.log.gz 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36906.log.gz 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35477.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35693.log.gz 2024-05-08T15:56:13.420 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38329.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38475.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34131.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35477.log.gz 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38150.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34237.log.gz 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34131.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38475.log.gz 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34274.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38329.log.gz 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34131.log.gz 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34167.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38150.log.gz 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35622.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35407.log 2024-05-08T15:56:13.421 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34274.log.gz 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35622.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34167.log.gz 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37578.log 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35407.log: gzip -5 87.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.35622.log.gz /var/log/ceph/ceph-client.admin.34452.log 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37184.log 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35407.log.gz 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34701.log 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37184.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34452.log.gz 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35095.log 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37578.log.gz 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35240.log 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35095.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37184.log.gz 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37685.log 2024-05-08T15:56:13.422 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34701.log.gz 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38909.log 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35240.log: /var/log/ceph/ceph-client.admin.37685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37935.log 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35095.log.gz 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35310.log 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37935.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35240.log.gz 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37013.log 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37685.log.gz 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35310.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38909.log.gz 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34987.log 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37013.log: 87.3% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35310.log.gz -- replaced with /var/log/ceph/ceph-client.admin.37935.log.gz 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36014.log 2024-05-08T15:56:13.423 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.g.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34987.log: /var/log/ceph/ceph-client.admin.36014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37899.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37013.log.gz 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.g.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35907.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34987.log.gz 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34810.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36014.log.gz 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35132.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37899.log.gz 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37399.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35907.log.gz 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38655.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37399.log: gzip 87.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.35132.log.gz -- 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.37722.log 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34810.log.gz 2024-05-08T15:56:13.424 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36870.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38655.log: /var/log/ceph/ceph-client.admin.37722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36693.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37399.log.gz 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36870.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37722.log.gz 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38655.log.gz 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38257.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34489.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36870.log.gz 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36693.log.gz 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37937.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35730.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34489.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38257.log.gz 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34703.log 2024-05-08T15:56:13.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35730.log: gzip -5 --verbose 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34489.log.gz 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr: -- /var/log/ceph/ceph-client.admin.36121.log 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34703.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37937.log.gz 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38981.log 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35730.log.gz 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37084.log 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38981.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34703.log.gz 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35623.log 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37084.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38981.log.gz 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34773.log 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% 96.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36121.log.gz 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37294.log 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31264.log 2024-05-08T15:56:13.426 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34773.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37084.log.gz 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37615.log 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35623.log.gz 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34773.log.gz 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31142.log 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.31264.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37294.log.gz 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38474.log 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31264.log.gz 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37256.log 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.31142.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37615.log.gz 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38474.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31142.log.gz 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37256.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T15:56:13.427 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38474.log.gz 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37828.log 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37256.log.gz 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37255.log 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34916.log 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35023.log 93.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34916.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37828.log.gz 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38113.log 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.34916.log.gz 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35023.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37255.log.gz 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36585.log 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38113.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35023.log.gz 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37793.log 2024-05-08T15:56:13.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37793.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38113.log.gz 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36264.log 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37793.log.gz 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37015.log: --verbose -- /var/log/ceph/ceph-client.admin.37185.log 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36585.log.gz 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37614.log 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36264.log.gz 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37614.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37185.log.gz 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38693.log 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5/var/log/ceph/ceph-client.admin.36159.log: --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T15:56:13.429 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38693.log: gzip -5 --verbose 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37614.log.gz 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr: -- /var/log/ceph/ceph-client.admin.36657.log 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.4.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38693.log.gz 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36371.log 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38980.log 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38149.log 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36657.log.gz 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36372.log 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35837.log 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38149.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36371.log.gz 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36372.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38980.log.gz 2024-05-08T15:56:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39349.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36372.log.gz 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38907.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38149.log.gz 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38799.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35837.log.gz 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34772.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38799.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39349.log.gz 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34168.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38907.log.gz/var/log/ceph/ceph-client.admin.34772.log: 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38799.log.gz 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36977.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.34168.log: -- /var/log/ceph/ceph-client.admin.34380.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36977.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.i.log 2024-05-08T15:56:13.431 INFO:teuthology.orchestra.run.smithi081.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34772.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35406.log 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34168.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34380.log: /var/log/ceph/ceph-mon.i.log: gzip 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.36907.log 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36977.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35406.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34380.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38798.log 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4%/var/log/ceph/ceph-client.admin.36907.log: -- replaced with /var/log/ceph/ceph-client.admin.35406.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37508.log 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36907.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39087.log 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36122.log 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39087.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38798.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37508.log.gz 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34488.log 2024-05-08T15:56:13.432 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36335.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39087.log.gz 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36122.log: /var/log/ceph/ceph-client.admin.34488.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36122.log.gz 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34488.log.gz 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36335.log: /var/log/ceph/ceph-client.admin.37401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36550.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35514.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 88.4% 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36335.log.gz 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34596.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38114.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34596.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37792.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36550.log.gz 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38258.log 2024-05-08T15:56:13.433 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38114.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34596.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37792.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38114.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31353.log 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34559.log 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37792.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38258.log: /var/log/ceph/ceph-client.admin.31353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39244.log 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34559.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31353.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38258.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34559.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37723.log 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37506.log 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39244.log: /var/log/ceph/ceph-client.admin.37723.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37506.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39244.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37723.log.gz 2024-05-08T15:56:13.434 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31222.log 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37506.log.gz 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34275.log 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.31222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36157.log 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34275.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31222.log.gz 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34275.log.gz 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35239.log 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34666.log 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35239.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35239.log.gz 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36157.log.gz 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.h.log 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34917.log 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.h.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34666.log.gz 2024-05-08T15:56:13.435 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31182.log 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34917.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34917.log.gz 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35944.log 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.31182.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31182.log.gz 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38007.log 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35944.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35944.log.gz 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38007.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38007.log.gz 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37293.log 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39351.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38438.log 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37293.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37293.log.gz 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2024-05-08T15:56:13.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38438.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38438.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38044.log 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37471.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36908.log 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38044.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38044.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35729.log 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36908.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36908.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39206.log 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35729.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35729.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38762.log 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39206.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39206.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38547.log 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38762.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38762.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38330.log 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38547.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38547.log.gz 2024-05-08T15:56:13.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34166.log 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38330.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38330.log.gz 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35408.log 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34166.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34166.log.gz 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35621.log 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35408.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35408.log.gz 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35942.log 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35621.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35621.log.gz 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37470.log 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35942.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35942.log.gz 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37186.log 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37470.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37470.log.gz 2024-05-08T15:56:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34273.log 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37186.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37186.log.gz 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35133.log 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34273.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34273.log.gz 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34025.log 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35133.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35133.log.gz 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37400.log 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34025.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34025.log.gz 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37400.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37400.log.gz 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35836.log 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38908.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gz 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37830.log 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35836.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35836.log.gz 2024-05-08T15:56:13.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34381.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37830.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37830.log.gz 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37364.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34381.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34381.log.gz 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37364.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37364.log.gz 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39313.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38006.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34026.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39313.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39313.log.gz 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34345.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34026.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.34026.log.gz 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35515.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34345.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34345.log.gz 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38583.log 2024-05-08T15:56:13.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35515.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35515.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36443.log 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38583.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38583.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36052.log 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36443.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36443.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36373.log 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36052.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36052.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36799.log 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36373.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36373.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39017.log 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36799.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36799.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38220.log 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39017.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39017.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37616.log 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38220.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38220.log.gz 2024-05-08T15:56:13.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34487.log 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37616.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37616.log.gz 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36694.log 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34487.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34487.log.gz 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38366.log 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36694.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36694.log.gz 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38437.log 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38366.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38366.log.gz 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36586.log 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38437.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38437.log.gz 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38800.log 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36586.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36586.log.gz 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38763.log 2024-05-08T15:56:13.442 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38800.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38800.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34702.log 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38763.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38763.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36051.log 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34702.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35692.log 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36051.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36051.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34238.log 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35692.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35692.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35800.log 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34238.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34238.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34059.log 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35800.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35800.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38585.log 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34059.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34059.log.gz 2024-05-08T15:56:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38151.log 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38585.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38585.log.gz 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35513.log 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38151.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38151.log.gz 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35513.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35513.log.gz 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37507.log 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37014.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37507.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37507.log.gz 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34130.log 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35943.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-05-08T15:56:13.444 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35906.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34130.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34130.log.gz 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.f.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35906.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35906.log.gz 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39137.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36050.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39137.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39137.log.gz 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36801.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36050.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36050.log.gz 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38869.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36801.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36801.log.gz 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38869.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38869.log.gz 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39350.log 2024-05-08T15:56:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36978.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37292.log 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39350.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39350.log.gz 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36265.log 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37292.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37292.log.gz 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34986.log 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36265.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36265.log.gz 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38870.log 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34986.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34986.log.gz 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36229.log 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38870.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38870.log.gz 2024-05-08T15:56:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38367.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36229.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36229.log.gz 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34061.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38367.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38367.log.gz 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38584.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34061.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34061.log.gz 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35728.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38584.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38584.log.gz 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38546.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35728.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35728.log.gz 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35131.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38546.log.gz 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36266.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35131.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35131.log.gz 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35022.log 2024-05-08T15:56:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36266.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36266.log.gz 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35585.log 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35022.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35022.log.gz 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35799.log 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35585.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35585.log.gz 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37363.log 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35799.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35799.log.gz 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34382.log 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37363.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37363.log.gz 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34594.log 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34382.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34382.log.gz 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37686.log 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34594.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34594.log.gz 2024-05-08T15:56:13.448 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39242.log 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37686.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37686.log.gz 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36656.log 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39242.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39242.log.gz 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36478.log 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36656.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36656.log.gz 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34451.log 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36478.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36478.log.gz 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34809.log 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34451.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34451.log.gz 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34060.log 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34809.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34809.log.gz 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39207.log 2024-05-08T15:56:13.449 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34060.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34060.log.gz 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36442.log 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39207.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39207.log.gz 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36587.log 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36442.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36442.log.gz 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39136.log 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36587.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36587.log.gz 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37579.log 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39136.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39136.log.gz 2024-05-08T15:56:13.450 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2024-05-08T15:56:13.455 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2024-05-08T15:56:13.455 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41131.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41131.log.gz 2024-05-08T15:56:13.455 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35695.log 2024-05-08T15:56:13.455 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40543.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2024-05-08T15:56:13.456 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35315.log 2024-05-08T15:56:13.456 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35695.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35105.log 2024-05-08T15:56:13.456 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35315.log: -- replaced with /var/log/ceph/ceph-client.admin.35695.log.gz 2024-05-08T15:56:13.462 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37579.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37579.log.gz 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35024.log 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38691.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38042.log 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35024.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35024.log.gz 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36871.log 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38042.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38042.log.gz 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37829.log 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36871.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36871.log.gz 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34879.log 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37829.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37829.log.gz 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36480.log 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34879.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34879.log.gz 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34915.log 2024-05-08T15:56:13.463 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36480.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36480.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37936.log 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.34915.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34915.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38656.log 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37936.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37936.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36692.log 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38656.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38656.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36158.log 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36692.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36692.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39018.log 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36158.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36158.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39243.log 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39018.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39018.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39088.log 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39243.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39243.log.gz 2024-05-08T15:56:13.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37721.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39088.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39088.log.gz 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38473.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.37721.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37721.log.gz 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36015.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38473.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38473.log.gz 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36479.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36015.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36015.log.gz 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39016.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36479.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36479.log.gz 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36764.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39016.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39016.log.gz 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39135.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.36764.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36764.log.gz 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35584.log 2024-05-08T15:56:13.465 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.39135.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39135.log.gz 2024-05-08T15:56:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38368.log 2024-05-08T15:56:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35584.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-05-08T15:56:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35478.log 2024-05-08T15:56:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.38368.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38368.log.gz 2024-05-08T15:56:13.466 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.35478.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35478.log.gz 2024-05-08T15:56:13.466 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38540.log 2024-05-08T15:56:13.467 INFO:teuthology.orchestra.run.smithi007.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35315.log.gz 2024-05-08T15:56:13.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35105.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35105.log.gz 2024-05-08T15:56:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39714.log 2024-05-08T15:56:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.38540.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38540.log.gz 2024-05-08T15:56:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.39714.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39714.log.gz 2024-05-08T15:56:13.521 INFO:teuthology.orchestra.run.smithi081.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.i.log.gz 2024-05-08T15:56:13.539 INFO:teuthology.orchestra.run.smithi007.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T15:56:13.539 INFO:teuthology.orchestra.run.smithi007.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.e.log.gz 2024-05-08T15:56:13.651 INFO:teuthology.orchestra.run.smithi081.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.h.log.gz 2024-05-08T15:56:13.654 INFO:teuthology.orchestra.run.smithi007.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.d.log.gz 2024-05-08T15:56:13.696 INFO:teuthology.orchestra.run.smithi081.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.g.log.gz 2024-05-08T15:56:13.703 INFO:teuthology.orchestra.run.smithi007.stderr: 90.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T15:56:13.705 INFO:teuthology.orchestra.run.smithi007.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T15:56:14.339 INFO:teuthology.orchestra.run.smithi081.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.f.log.gz 2024-05-08T15:56:15.469 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T15:56:15.478 INFO:teuthology.orchestra.run.smithi007.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T15:56:15.487 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T15:56:15.535 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T15:56:15.537 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-05-08T15:56:15.537 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m2.384s 2024-05-08T15:56:15.537 INFO:teuthology.orchestra.run.smithi081.stderr:user 0m9.276s 2024-05-08T15:56:15.538 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m0.460s 2024-05-08T15:56:15.610 INFO:teuthology.orchestra.run.smithi007.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T15:56:15.966 INFO:teuthology.orchestra.run.smithi007.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T15:56:15.968 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-05-08T15:56:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m2.835s 2024-05-08T15:56:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m9.491s 2024-05-08T15:56:15.968 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m0.446s 2024-05-08T15:56:15.968 INFO:tasks.ceph:Archiving logs... 2024-05-08T15:56:15.968 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/remote/smithi007/log 2024-05-08T15:56:15.969 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T15:56:17.616 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/remote/smithi081/log 2024-05-08T15:56:17.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T15:56:19.304 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T15:56:19.314 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T15:56:19.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T15:56:19.317 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T15:56:19.393 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T15:56:19.401 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T15:56:19.402 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T15:56:19.404 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T15:56:19.420 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-05-08T15:56:19.421 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-05-08T15:56:19.425 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T15:56:19.425 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-05-08T15:56:19.425 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 -1397us[-1387us] +/- 49ms 2024-05-08T15:56:19.425 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 +336us[ +346us] +/- 55ms 2024-05-08T15:56:19.425 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +1737us[+1747us] +/- 72ms 2024-05-08T15:56:19.425 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:56:19.426 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T15:56:19.427 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-05-08T15:56:19.427 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -1398us[-1392us] +/- 49ms 2024-05-08T15:56:19.427 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 +332us[ +332us] +/- 55ms 2024-05-08T15:56:19.427 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 +1727us[+1727us] +/- 72ms 2024-05-08T15:56:19.427 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T15:56:19.427 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T15:56:19.436 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T15:56:19.437 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T15:56:19.445 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T15:56:19.540 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T15:56:19.608 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T15:56:19.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:56:19.640 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T15:56:19.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:56:19.669 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T15:56:19.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T15:56:19.690 INFO:teuthology.task.internal:Duration was 1030.258777 seconds 2024-05-08T15:56:19.690 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T15:56:19.700 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T15:56:19.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T15:56:19.703 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T15:56:19.744 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T15:56:19.749 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T15:56:20.128 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T15:56:20.128 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-05-08T15:56:20.128 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T15:56:20.194 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-05-08T15:56:20.194 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T15:56:20.218 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T15:56:20.218 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T15:56:20.237 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T15:56:20.285 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T15:56:20.285 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T15:56:20.306 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T15:56:20.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T15:56:20.435 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T15:56:20.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T15:56:20.461 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T15:56:20.486 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T15:56:20.496 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T15:56:20.504 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T15:56:20.525 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-05-08T15:56:20.549 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-05-08T15:56:20.571 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T15:56:20.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:56:20.608 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T15:56:20.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T15:56:20.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T15:56:20.637 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T15:56:20.638 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/remote/smithi007 2024-05-08T15:56:20.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T15:56:20.686 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697604/remote/smithi081 2024-05-08T15:56:20.687 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T15:56:20.720 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T15:56:20.720 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T15:56:20.724 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T15:56:20.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T15:56:20.781 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T15:56:20.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T15:56:20.833 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T15:56:20.833 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T15:56:20.835 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T15:56:20.848 INFO:teuthology.orchestra.run.smithi007.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 15:56 /home/ubuntu/cephtest 2024-05-08T15:56:20.850 INFO:teuthology.orchestra.run.smithi081.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 15:56 /home/ubuntu/cephtest 2024-05-08T15:56:20.851 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T15:56:20.862 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T15:56:20.885 INFO:teuthology.run:Summary data: description: rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_latest} thrashers/sync-many workloads/pool-create-delete} duration: 1030.258777141571 flavor: default owner: scheduled_lflores@teuthology success: true 2024-05-08T15:56:20.885 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T15:56:21.043 INFO:teuthology.run:pass